update some more docs

Signed-off-by: strawberry <strawberry@puppygock.gay>
This commit is contained in:
strawberry 2024-12-15 22:46:56 -05:00
parent 7ad710d96a
commit 8dcdb4b09a
No known key found for this signature in database
6 changed files with 93 additions and 33 deletions

2
debian/README.md vendored
View file

@ -9,6 +9,8 @@ It is recommended to see the [generic deployment guide](../deploying/generic.md)
for further information if needed as usage of the Debian package is generally
related.
No `apt` repository is currently offered yet, it is in the works/development.
### Configuration
When installed, the example config is placed at `/etc/conduwuit/conduwuit.toml`

View file

@ -24,6 +24,9 @@ OCI images for conduwuit are available in the registries listed below.
[shield-latest]: https://img.shields.io/docker/image-size/girlbossceo/conduwuit/latest
[shield-main]: https://img.shields.io/docker/image-size/girlbossceo/conduwuit/main
OCI image `.tar.gz` files are also hosted directly at when uploaded by CI with a
commit hash/revision or a tagged release: <https://pup.systems/~strawberry/conduwuit/>
Use
```bash

View file

@ -1,6 +1,6 @@
# Generic deployment documentation
> ## Getting help
> ### Getting help
>
> If you run into any problems while setting up conduwuit, ask us in
> `#conduwuit:puppygock.gay` or [open an issue on
@ -8,29 +8,50 @@
## Installing conduwuit
You may simply download the binary that fits your machine. Run `uname -m` to see
what you need.
### Static prebuilt binary
You may simply download the binary that fits your machine architecture (x86_64
or aarch64). Run `uname -m` to see what you need.
Prebuilt fully static musl binaries can be downloaded from the latest tagged
release [here](https://github.com/girlbossceo/conduwuit/releases/latest) or
`main` CI branch workflow artifact output. These also include Debian/Ubuntu packages.
`main` CI branch workflow artifact output. These also include Debian/Ubuntu
packages.
Binaries are also available on my website directly at: <https://pup.systems/~strawberry/conduwuit/>
These can be curl'd directly from. `ci-bins` are CI workflow binaries by commit
hash/revision, and `releases` are tagged releases. Sort by descending last
modified for the latest.
These binaries have jemalloc and io_uring statically linked and included with
them, so no additional dynamic dependencies need to be installed.
For the **best** performance; if using an `x86_64` CPU made in the last ~15 years,
we recommend using the `-haswell-` optimised binaries. This sets
`-march=haswell` which is the most compatible and highest performance with
optimised binaries. The database backend, RocksDB, most benefits from this as it
will then use hardware accelerated CRC32 hashing/checksumming which is critical
for performance.
### Compiling
Alternatively, you may compile the binary yourself. We recommend using
Nix (or [Lix](https://lix.systems)) to build conduwuit as this has the most guaranteed
reproducibiltiy and easiest to get a build environment and output going. This also
allows easy cross-compilation.
Nix (or [Lix](https://lix.systems)) to build conduwuit as this has the most
guaranteed reproducibiltiy and easiest to get a build environment and output
going. This also allows easy cross-compilation.
You can run the `nix build -L .#static-x86_64-linux-musl-all-features` or
`nix build -L .#static-aarch64-linux-musl-all-features` commands based
on architecture to cross-compile the necessary static binary located at
`result/bin/conduwuit`. This is reproducible with the static binaries produced in our CI.
`result/bin/conduwuit`. This is reproducible with the static binaries produced
in our CI.
Otherwise, follow standard Rust project build guides (installing git and cloning
the repo, getting the Rust toolchain via rustup, installing LLVM toolchain +
libclang for RocksDB, installing liburing for io_uring and RocksDB, etc).
If wanting to build using standard Rust toolchains, make sure you install:
- `liburing-dev` on the compiling machine, and `liburing` on the target host
- LLVM and libclang for RocksDB
You can build conduwuit using `cargo build --release --all-features`
## Migrating from Conduit

View file

@ -29,6 +29,12 @@ conduit:Isq8FGyEC6FOXH6nD+BOeAA+bKp6X6UIbupSlGEPuOg=
conduwuit:lYPVh7o1hLu1idH4Xt2QHaRa49WRGSAqzcfFd94aOTw=
```
If needed, we have a binary cache on Cachix but it is only limited to 5GB:
```
https://conduwuit.cachix.org
conduwuit.cachix.org-1:MFRm6jcnfTf0jSAbmvLfhO3KBMt4px+1xaereWXp8Xg=
```
If specifying a Git remote URL in your flake, you can use any remotes that
are specified on the README (the mirrors), such as the GitHub: `github:girlbossceo/conduwuit`

View file

@ -22,23 +22,59 @@ conduwuit has moderation admin commands for:
Any commands with `-list` in them will require a codeblock in the message with
each object being newline delimited. An example of doing this is:
```` !admin rooms moderation ban-list-of-rooms ``` !roomid1:server.name
!roomid2:server.name !roomid3:server.name ``` ````
````
!admin rooms moderation ban-list-of-rooms
```
!roomid1:server.name
#badroomalias1:server.name
!roomid2:server.name
!roomid3:server.name
#badroomalias2:server.name
```
````
## Database
## Database (RocksDB)
If using RocksDB, there's very little you need to do. Compaction is ran
automatically based on various defined thresholds tuned for conduwuit to be high
performance with the least I/O amplifcation or overhead. Manually running
compaction is not recommended, or compaction via a timer. RocksDB is built with
io_uring support via liburing for async read I/O.
Generally there is very little you need to do. [Compaction][rocksdb-compaction]
is ran automatically based on various defined thresholds tuned for conduwuit to
be high performance with the least I/O amplifcation or overhead. Manually
running compaction is not recommended, or compaction via a timer, due to
creating unnecessary I/O amplification. RocksDB is built with io_uring support
via liburing for improved read performance.
RocksDB troubleshooting can be found [in the RocksDB section of troubleshooting](troubleshooting.md).
### Compression
Some RocksDB settings can be adjusted such as the compression method chosen. See
the RocksDB section in the [example config](configuration/examples.md). btrfs
users may benefit from disabling compression on RocksDB if CoW is in use.
the RocksDB section in the [example config](configuration/examples.md).
RocksDB troubleshooting can be found [in the RocksDB section of
troubleshooting](troubleshooting.md).
btrfs users have reported that database compression does not need to be disabled
on conduwuit as the filesystem already does not attempt to compress. This can be
validated by using `filefrag -v` on a `.SST` file in your database, and ensure
the `physical_offset` matches (no filesystem compression). It is very important
to ensure no additional filesystem compression takes place as this can render
unbuffered Direct IO inoperable, significantly slowing down read and write
performance. See <https://btrfs.readthedocs.io/en/latest/Compression.html#compatibility>
> Compression is done using the COW mechanism so its incompatible with
> nodatacow. Direct IO read works on compressed files but will fall back to
> buffered writes and leads to no compression even if force compression is set.
> Currently nodatasum and compression dont work together.
### Files in database
Do not touch any of the files in the database directory. This must be said due
to users being mislead by the `.log` files in the RocksDB directory, thinking
they're server logs or database logs, however they are critical RocksDB files
related to WAL tracking.
The only safe files that can be deleted are the `LOG` files (all caps). These
are the real RocksDB telemetry/log files, however conduwuit has already
configured to only store up to 3 RocksDB `LOG` files due to generall being
useless for average users unless troubleshooting something low-level. If you
would like to store nearly none at all, see the `rocksdb_max_log_files`
config option.
## Backups
@ -95,3 +131,5 @@ Built-in S3 support is also planned, but for now using a "S3 filesystem" on
`media/` works. conduwuit also sends a `Cache-Control` header of 1 year and
immutable for all media requests (download and thumbnail) to reduce unnecessary
media requests from browsers, reduce bandwidth usage, and reduce load.
[rocksdb-compaction]: https://github.com/facebook/rocksdb/wiki/Compaction

View file

@ -91,16 +91,6 @@ reliability at a slight performance cost due to TCP overhead.
## RocksDB / database issues
#### Direct IO
Some filesystems may not like RocksDB using [Direct
IO](https://github.com/facebook/rocksdb/wiki/Direct-IO). Direct IO is for
non-buffered I/O which improves conduwuit performance and reduces system CPU
usage, but at least FUSE and possibly ZFS are filesystems potentially known
to not like this. See the [example config](configuration/examples.md) for
disabling it if needed. Issues from Direct IO on unsupported filesystems are
usually shown as startup errors.
#### Database corruption
If your database is corrupted *and* is failing to start (e.g. checksum