You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
eiknat 9809fa9b3e Merge pull request 'user: add user details endpoint' (#36) from feature/user-details into main 7 months ago
src user: move some stuff around 7 months ago
.eslintignore fix deps/eslintconfig 9 months ago
.eslintrc.json user: add user details endpoint 7 months ago
.gitignore users: add user_view, make routes modular, add relations 9 months ago
CODE_OF_CONDUCT.md init: stub out more models/controllers/middleware 9 months ago
CONTRIBUTING.md init: stub out more models/controllers/middleware 9 months ago
LICENSE init: stub out more models/controllers/middleware 9 months ago
README.md Update 'README.md' 8 months ago
package.json user: hash password, generate keypair 8 months ago
tsconfig.json fix deps/eslintconfig 9 months ago
yarn.lock user: hash password, generate keypair 8 months ago

README.md

Hexbear Typescript

A quick sample repo of how a rewritten backend in typescript could look using:

In consideration but not shown here:

Getting Started

If you have not already installed typescript as part of working on the hexbear/lemmy frontend or are new to node:

Install NVM
NVM Repo

You can follow the instructions on their repo to also get the latest version of node for your OS of choice.

Install Yarn
Yarn

Yarn is the package manager we already use so it's suggested here.

Install TypeScript
yarn global add typescript

Install Knex
This step is optional for if you want to run migrations/seeders through the knex cli, it's fine to skip it of you dont want to do this step
yarn global add knex

PostgreSQL
EnterpriseDB has installers for windows/osx/linux that will come with some management tools for qol. We currently use version 12.
Docker Official docker if you prefer that route, just make sure to update src/database/knexfile.ts with the correct ip address/port.
pgAdmin Cross platform GUI tool to manage postgresql databases. This comes with the EnterpriseDB installers. If you're not comfortable with using psql this is the recommended route.

Using psql or pgAdmin we want to:

  • Create a user hexbear with password hexbear
  • Create a database hexbear with owner hexbear
  • Make a schema called public if it does not exist
  • Make a schema called knex, this wont exist by default. We will store our knex migrations there so they arent affected by operations involving the main schemas.

The above matches what we use in src/database/knexfile, if you have a preferred local setup then you can adjust the knexfile. But be warned that the initial migration assumes an empty database with a user hexbear so adjust accordingly.

Migrations/Seeders
To create a migration: yarn run knex:migration-make migration_name_here
To run all pending migrations: yarn run knex:migration-run
To create a seeder file: yarn run knex:seed-make seed_name_here
To run all seeds: yarn run knex:seed-run

To handle running specific migrations or seeds, the same scripts can be used but you'll need to specify the file name.
For migrations: knex migrate:(up|down) file_name
seeds: knex seed:run --specific=file_name

Starting the App

  • Clone this repo and run yarn
  • Run yarn run knex:migration-run to build your database
  • (Optionally) run yarn run knex:seed-run to populate your database
  • Start the server with yarn run dev

Go to http://localhost:8080 to play pong or http://localhost:8080/api/v1/users to see sample data!

Other Tools

IDE/Editors
VSCode - Has great TS support, autocomplete, is open source (but MS backed), cross platform and one of the more popular editors out there

DB/Querying
Postico - OSX postgresql app. Has a slicker UI and is easier to use than pgAdmin but mainly built for querying and lacks management options
DBeaver - Cross platform, good name, probably runs better than pgAdmin

Goals / Why Rewrite / Why Typescript?

The current backend has a lot of Technical-Debt™; we've made several critical changes that have let us scale to 10k+ users and had to put out a lot of fires on the way. These changes have also left us in a state where merging upstream Lemmy is a large job even if we would do it weekly to keep up.

We as a team feel we have hit a dead-end or brick wall with continuing to maintain the current backend in rust as-is while maintaining upstream compatibility. Additionally, there are several architectural changes we would like to make that would both require forking and a significant rewrite of the current codebase.

Since a rewrite is inevitable and rust/choice rust libs have shortcomings or simply aren't ready for maintaining a production load web server, we have decided to use a new language. On the human side, the "core team" feels that the current tech stack is painful or slow to work with, makes onboarding difficult, and has been a turn off from getting new contributors. So-

Goals

In the short/mid term we want to:

  • Convert majority of client requests to the rest api away from websockets
  • Build a v2 api with the goal of splitting the current request/response objects up (ie. make it possible to query smaller data sets so a User component isn't getting irrelevant data, basically make an actual modern/"standard" api)
  • Use a mature query builder that allows rewriting the sql queries/joins to be sensible, not reliant on cross joins, can properly filter based on inputs etc

Why Rewrite?

In technical specifics

  • Rust compile times are frustrating
  • Many Rust libs are not mature enough for a small team web server to be a good idea, nor is the query building/orm side without strict limitations

In human terms

  • Some of us are facing severe burnout on Rust or dont find it fun for this project's specifics
  • We're finding it difficult to keep contributing fixes because we have to code in a reactionary style and Rust does not lend itself to getting something quick done
  • Rust learning curve is steep or the lang itself turns away new contributors

Why Typescript?

Technical:

  • It offers typing/auto-complete/other goodies to JS
  • The ecosystem has many mature libs web servers, middleware, querying, testing, documentation
  • Iteration is quick and our BE/FE tech stack would be similar
  • We do not face any optimization/performance issues that make compiled langs needed

Human:

  • Most of us are familiar with TS/JS already
  • Learning curve is small and we can maintain a codebase that is friendly to new devs or even someone new to coding
  • The TS community is very friendly, welcoming and focused on accessibility. It's extremely fast growing and immensely popular for web projects- As such it offers the best chance to sustain the project via new members

But most importantly, we want this to be fun, even if that means putting in the work to fork and rewrite in a new language. There will be new pain points, new tech debt, and many human hours of work to even bring this to current 1:1 functionality, but that is worth it if the end result is something we own, enjoy and can easily share.

Thinking Ahead

Since this is just a showcase in "how it looks" and "why", we still have to decide on some things:

  • What design pattern(s) do we want to use?
  • How should we split up work / what areas should we focus on first?
  • How do we want the repo structured?
  • What other libs do we need?