Back in the days of old, before this LLM shit took off, discovering a new project was pleasant. Someone would post a new Emacs package, or a new Haskell project on Reddit or Discourse, I'd click the link and then scroll pleasantly through the code. I would spend a couple of minutes reading code and enjoying myself. I'd peruse the repository, both learning and also trying to guess the skill level of the developer behind it. At the end of my reading session, I'd then decide if I wanted to use the project or not. Things were simple back then.

But then LLMs came along and people got upset - in my opinion for good reason - and so they starting hating on anyone and anything even remotely touched by an LLM. The natural and expected response to that is LLM proponents hiding the fact that they use LLMs. In effect, lying to me.

These days, discovering a new project is a chore, because first things first, I have to answer the unpleasant question:

Is this person trying to lie to me?

I don't enjoy second-guessing everything, I want to write software, share it freely with others and play with computers. Not be a human LLM detector.

Please stop hiding that you're using LLMs, I think most people are tired enough where they won't lynch you for using an LLM. Honestly disclosing LLM use allows me to skip your project, or look at it later when I have the mental capacity to look for extremely subtle bugs that you didn't catch.

Side note, I personally think that when you post a project on Reddit/Discourse the post itself should declare that you used an LLM. We all know that LLMs are a touchy subject and as such I find it disrespectful when you don't do that. Knowing in advance what to expect lowers my disappointment and as said before, allows me to focus my energy on actually reading the code and not trying to detect LLM use.

Too Many Datasets

Given a situation where a ZFS pool has just too many datasets for you to comfortably manage, or perhaps you have a few datasets, but you just learned of a property that you really should have set from the start, what do you do? Well, I don't know what you do, I would love to hear about that, so please do reach out to me, over Matrix preferably.

In any case, what I came up with is disko-zfs. A simple Rust program that will declaratively manage datasets on a zpool. It does this based on a JSON specification, which lists the datasets, their properties and a few pieces of extra information.

The Schema

{
  "datasets": {
    "zroot": {
      "properties": {
        "atime": "off",
        "com.sun:auto-snapshot": "false",
        "compression": "zstd-2",
        "dnodesize": "auto",
        "mountpoint": "none",
        "recordsize": "128K",
        "xattr": "on"
      }
    },
    ...
    "zroot/ds1/persist/var/lib/forgejo": {
      "properties": {
        "mountpoint": "legacy"
      }
    },
    "zroot/ds1/persist/var/lib/postgresql": {
      "properties": {
        "mountpoint": "legacy",
        "recordsize": "8k"
      }
    },
    ...
  },
  "ignoredDatasets": [
    "zroot/ds1/root/*"
  ],
  "ignoredProperties": [
    "nixos:shutdown-time",
    ":generation",
    "com.sun:auto-snapshot"
  ],
  "logLevel": "info"
}

As you can see, it's relatively self explanatory. The information that this JSON format carries is:

an attribute set of datasets and their properties a list of ignored datasets that disko-zfs will never create, modify, or destroy a list of ignored properties that disko-zfs will never create, modify, or delete the loglevel at which disko-zfs logs

With the schema in hand, we can execute disko-zfs apply --file spec.json on a live machine, where we get the following output:

# !! Destructive Commands !!
> zfs destroy zroot/ds1/persist/var/lib/gitea
# Applying...
+ zfs set mountpoint=legacy zroot/ds1/persist/var/lib/forgejo
+ zfs create -orecordsize=8k -omountpoint=legacy zroot/ds1/persist/var/lib/forgejo
# Done!

It tells us a few things:

that zfs destroy zroot/ds1/persist/var/lib/gitea would be executed, but it wasn't that the following two commands have been executed: zfs set mountpoint=legacy zroot/ds1/persist/var/lib/forgejo zfs create -orecordsize=8k -omountpoint=legacy zroot/ds1/persist/var/lib/forgejo

If we run disko-zfs apply --file spec.json again on the same machine, disko-zfs will correctly report that there is nothing to do.

Instead of directly applying the configuration we could have used the plan subcommand to tell disko-zfs just tell us what it would do, if we executed apply, even for non-destructive commands. This is a good way of verifying that disko-zfs won't do anything unexpected.

NixOS Integration

If you also happen to be a NixOS user, then this section will be of interest to you. disko-zfs supports NixOs natively; it exposes a NixOS module, which adds the disko.zfs option group. It importantly adds the disko.zfs.settings option, whose structure is isomorphic to the structure used by the disko-zfs program itself; as such we can directly translate the previously shown JSON to Nix and everything will work as expected, neat!

{
  disko.zfs = {
    enable = true;
    settings = {
      datasets = {
        "zroot" = {
          properties = {
            atime = "off";
            "com.sun:auto-snapshot" = "false";
            compression = "zstd-2";
            dnodesize = "auto";
            mountpoint = "none";
            recordsize = "128K";
            xattr = "on";
          };
        };
        ...
        "zroot/ds1/persist/var/lib/forgejo" = {
          properties = {
            mountpoint = "legacy";
          };
        };
        "zroot/ds1/persist/var/lib/postgresql" = {
          properties = {
            mountpoint = "legacy";
            recordsize = "8k";
          };
        };
        ...
      };
      ignoredDatasets = [
        "zroot/ds1/root/*"
      ];
      ignoredProperties = [
        "nixos:shutdown-time"
        ":generation"
        "com.sun:auto-snapshot"
      ];
      logLevel = "info";
    };
  };
}

Given the above NixOS configuration, next time you run nixos-rebuild switch --flake .#your-server, disko-zfs will make any changes necessary to bring your zpool into shape. You can also execute nixos-rebuild dry-activate --flake .#your-server instead, which will cause disko-zfs to run in plan mode and merely print out the changes it would make, nifty way to test your changes.

YouTube Last year, while talking to a great friend of mine about addictions, he mentioned that making things inconvenient for oneself is a good way to stop doing them. I started off by DNS blocking YouTube domains on all my devices, the reason was that I was addicted to watching speedruns and letting YouTube lull me into a state of mindless video consumption. I could still access YouTube if I wanted to through alternative frontends like Invidious or Piped, but let's be frank, the UX of those slightly sucks. Surprisingly though their bad UX is a good thing. As expected I stopped frequenting YouTube and began doing other things, like watching TV shows, programming, actually playing the games I was watching people play online. In the end this experiment turned out to be a great success.

Discord Next I wanted to tackle Discord. Current alternative Discord clients focus on making the Discord experience better, but I weirdly wanted a worse one. So I looked at terminal clients briefly, but those are hard to use, don't display images and don't really work well on mobile. By that point I've already setup Matrix and have been planning to use it to bridge services like Slack and Facebook Messenger that I don't like the UX of anyway.

So I bridged Discord through Matrix and joined only a select few Discord servers on a completely new Discord account. As expected the time spent on Discord went down a lot, I stopped idly chatting with people and looking at memes. My overall life quality went up and then later I expanded my new life style even more by limiting online communication in general.

Moral of the Rambling What I wanted to get across using this post is, that making things inconvenient for oneself works great if one wishes to control how often they do said things. This applies to gaming, YouTube, social media and many more things I personally cannot vouch for. It does of course require sacrifices and a rather large amount of time depending on what technical solutions one decides to adopt.

I would also like to warn that it might lead to a slight disconnect from current events and from your friends. An overwhelming majority of people frequent Discord and other social media, especially younger folk, which means that if one disconnects from such platforms they loose out on a significant portion of socializing their peers do.

A good way to combat feelings of loneliness that may arise from radical changes in ones life is to realize how people lived in the past and to cherish real world interactions much more. Each conversation with a friend is a gift and it is imperative that it is taken seriously and thoroughly enjoyed.

Reasoning

I can speak from my own experience that discussing hard topics is easier through text and even easier in a non-native tongue (English). Talking about serious topics in person is almost never done, if it is, it doesn't go well. When I was talking to my friends over Discord and we started talking about something serious we'd instinctively switch to English.

Many people, me included, don't want to leave the house for a multitude of different reasons. It may be some random onset of social awkwardness, the feeling that everyone outside has nothing better to do than watch you and judge you (I have that too sometimes and it's the silliest thing ever). This then leads to unsocial weekends as generally during the week you're forced to leave the house, be it university or work. These solitary weekends then have negative consequences as one generally realizes that such a weekend is not good for them, which leads to feeling worse which leads to not leaving the house more. This starts a vicious cycle, but we'll come back to that later.

There are also those that have grown accustomed to using "the social battery" as an excuse to not be social. They decide they have a set amount of socializing they're able to do per day or week. If they go over that arbitrary amount, they start to manifest the feelings of being socially exhausted and then proceed to end the session. Interestingly, chatting online seems to deplete "the battery" less. Which makes sense, I'm not saying that this behavior is completely self-induced and made up, but I do think people that function like this, reinforce it within themselves. The fact that chatting online seems to not be as draining leads to such people relying on online communication to not feel alone while not going over their quotas. This leads to, what I would describe as a fake feeling of companionship.

This last point is applies to me personally. I seem to not be able to be alone, not because I need people 24/7, but because I don't seem to be able to be alone with just myself. I think it's a case of distracting myself with others, to not have to deal with me and how I feel. When the distraction goes away, I have no choice but to focus on myself again which then leads to feeling of sadness and such stuff. I have not observed this phenomenon in other people, but frankly this is a thing which manifests while alone and understandably people are reluctant to talk about it. As talking about it breaks the illusion that the distraction of socializing provides.

It is also interesting that people that live in huge cities feel more alone than those that live in small towns and villages. One of the causes is in my opinion the fact that in such a huge city everyone lives somewhere completely different so it's impractical to meet up randomly and on short notice. Meetings are arranged before-hand and a good while before they take place. While going to and from a social hangout, one is completely alone, even when surrounded by dozens of people on the metro, tram, bus or even while just walking home. The density of people one actually interacts with goes down, the rest may as well just be NPCs.

Effect

Due to all described above, I've decided that I will only use online communication mediums like Discord, Whatsapp, Matrix to arrange meetings and other real life events. I want to hang out with real people in the real world not with profile pictures on my phone. I've spent way too much time chatting with people, even though I care about those people, If I'm to continue interacting with them they'll have to be willing to meet up with me. I am more than willing to that and I hope others will be willing to do the same. If not then it is a shame but I will not stare into my phone alone. I'm sure some will be unhappy about this new policy, I'm sure I will loose a few friends, but my hope is that this change will lead to better friendships and a better well-being for me and my remaining friends. I refuse to be a part of and contribute to the degradation of humanity due to the internet and its less than amazing aspects.

Searx is a free internet metasearch engine which aggregates results from more than 70 search services. Users are neither tracked nor profiled. Additionally, searx can be used over Tor for online anonymity.

So if you're a privacy nerd or want ensure Google doesn't know what you're cooking tonight, read on and you'll learn how Searx works from a system administrator and packager perspective.

Searx is already packaged in nixpkgs, but for the sake of this blog post, let's pretend it isn't. I'll go over all the things I check, verify and all the things I do when packaging. So I'll quit mumbling and start Nix-ing!

Discovery

First it's imperative that we find the upstream repo we'll be working with, it may sound simple enough, and in the case of Searx it luckily is, but it can also be challenging. It all depends on how well-known the project is and how unique the name is. My recommendation is to use a search engine and search for searx git in this case, which gets us https://github.com/searx/searx.

Now that we have a link to the repo, we need to identify the language and in the case of some languages the build system. There are several ways to do this, one is to just look at the root of the repo and look for a few recognizable files. I'll leave an incomplete table below.

I won't list tools to use when packaging these different languages, because the recommended set changes often and I'd have to keep this blog post up to date :), but it's easy enough to search for them. Generally if you search for <package-manager>2nix.

Looking at the repository we see a requirements.txt and a setup.py, the first one is valuable because we should have a list of all python packages we need and the second we need to keep in mind, since it contains custom arbitrary python that we may need inspect and fix.

certifi==2022.5.18.1
babel==2.9.1
flask-babel==2.0.0
flask==2.1.1
jinja2==3.1.2
lxml==4.9.0
pygments==2.8.0
python-dateutil==2.8.2
pyyaml==6.0
httpx[http2]==0.23.0
Brotli==1.0.9
uvloop==0.16.0; python_version >= '3.7'
uvloop==0.14.0; python_version < '3.7'
httpx-socks[asyncio]==0.7.4
langdetect==1.0.9
setproctitle==1.2.2

It's also worth looking at the Dockerfile and any Makefile, Justfile, or scripts folder. Here we have a Dockerfile and also a Makefile, lucky! Let's start with the Dockerfile, I'll pick out the important bits only.

FROM alpine:3.15

A pretty crucial piece of information here, we now know both the distro the container uses so we can descern the environment a bit and that Searx will happily run on musl libc.

ENTRYPOINT ["/sbin/tini","--","/usr/local/searx/dockerfiles/docker-entrypoint.sh"]

Here we see where we should look for the startup script.

ENV INSTANCE_NAME=searx \
    AUTOCOMPLETE= \
    BASE_URL= \
    MORTY_KEY= \
    MORTY_URL= \
    SEARX_SETTINGS_PATH=/etc/searx/settings.yml \
    UWSGI_SETTINGS_PATH=/etc/searx/uwsgi.ini

Here we have a incomplete list of arguments we can pass to into the Docker container, it's important to later notice where they're handled, in the scripting or in the actual program itself?

apk add --no-cache -t build-dependencies \
  build-base \
  py3-setuptools \
  python3-dev \
  libffi-dev \
  libxslt-dev \
  libxml2-dev \
  openssl-dev \
  tar \
  git \

Here we see a list of packages installed with apt, but you (and me actually) may not know what does -t build-dependencies do. It's best to look at the manpage for apk add, so search for apk-add man. According to https://www.mankier.com/8/apk-add -t adds a virtual package with the dependencies listed on the command line and then installs that package. So we have one package build-dependencies containing a set of packages we need at build time.

apk add --no-cache \
  ca-certificates \
  su-exec \
  python3 \
  py3-pip \
  libxml2 \
  libxslt \
  openssl \
  tini \
  uwsgi \
  uwsgi-python3 \
  brotli \

Next we have a list of packages needed at runtime, this one is really important to remember since we may have to add these in a special way later. You'll see what I mean.

pip3 install --upgrade pip wheel setuptools \

Then it upgrades pip, wheel, and setuptools. I personally had to look up what wheel is. But looking at Alpine Linux packages yields no results, so let's just ignore it for now. If it doesn't come up later it's not important.

pip3 install --no-cache -r requirements.txt \

Second to last it installs the packages specied in requirements.txt as expected.

apk del build-dependencies \
&& rm -rf /root/.cache

And lastly it does some cleanup. Which is interesting, because I expected those dependencies to be used later by some custom searx native component, but I guess it makes sense they're not.

COPY searx ./searx
COPY dockerfiles ./dockerfiles

We now see where that startup script comes from.

RUN /usr/bin/python3 -m compileall -q searx; \
    touch -c --date=@${TIMESTAMP_SETTINGS} searx/settings.yml; \
    touch -c --date=@${TIMESTAMP_UWSGI} dockerfiles/uwsgi.ini; \
    if [ ! -z $VERSION_GITCOMMIT ]; then\
      echo "VERSION_STRING = VERSION_STRING + \"-$VERSION_GITCOMMIT\"" >> /usr/local/searx/searx/version.py; \
    fi; \
    find /usr/local/searx/searx/static -a \( -name '*.html' -o -name '*.css' -o -name '*.js' \
    -o -name '*.svg' -o -name '*.ttf' -o -name '*.eot' \) \
    -type f -exec gzip -9 -k {} \+ -exec brotli --best {} \+

This is a complicated little beast, we see searx/settings.yml dockerfiles/uwsgi.ini and /usr/local/searx/searx/version.py, we also see that it compiles all the python files, but that will be taken care of by nixpkgs. Interestingly it also compresses all the assets with gzip. The find command looks for all files with .html, .css, .js, .svg, .ttf and .eot, then executes gzip -9 -k and brotli --best. (here I had to again search for what's brotli). (it looks to be a compression scheme)

That's all from the Dockerfile. Now we need to look at the script it calls.

printf "\nEnvironment variables:\n\n"
printf "  INSTANCE_NAME settings.yml : general.instance_name\n"
printf "  AUTOCOMPLETE  settings.yml : search.autocomplete\n"
printf "  BASE_URL      settings.yml : server.base_url\n"
printf "  MORTY_URL     settings.yml : result_proxy.url\n"
printf "  MORTY_KEY     settings.yml : result_proxy.key\n"
printf "  BIND_ADDRESS  uwsgi bind to the specified TCP socket using HTTP protocol. Default value: \"${DEFAULT_BIND_ADDRESS}\"\n"

# update settings.yml
sed -i -e "s|base_url : False|base_url : ${BASE_URL}|g" \
   -e "s/instance_name : \"searx\"/instance_name : \"${INSTANCE_NAME}\"/g" \
   -e "s/autocomplete : \"\"/autocomplete : \"${AUTOCOMPLETE}\"/g" \
   -e "s/ultrasecretkey/$(openssl rand -hex 32)/g" \
   "${CONF}"

sed -i -e "s/image_proxy : False/image_proxy : True/g" \
            "${CONF}"
cat >> "${CONF}" <<-EOF

# Morty configuration
result_proxy:
   url : ${MORTY_URL}
   key : !!binary "${MORTY_KEY}"
EOF

if [ -f "${CONF}" ]; then
    if [ "${REF_CONF}" -nt "${CONF}" ]; then
        # There is a new version
        if [ $FORCE_CONF_UPDATE -ne 0 ]; then
            # Replace the current configuration
            printf '⚠️  Automaticaly update %s to the new version\n' "${CONF}"
            if [ ! -f "${OLD_CONF}" ]; then
                printf 'The previous configuration is saved to %s\n' "${OLD_CONF}"
                mv "${CONF}" "${OLD_CONF}"
            fi
            cp "${REF_CONF}" "${CONF}"
            $PATCH_REF_CONF "${CONF}"
        else
            # Keep the current configuration
            printf '⚠️  Check new version %s to make sure searx is working properly\n' "${NEW_CONF}"
            cp "${REF_CONF}" "${NEW_CONF}"
            $PATCH_REF_CONF "${NEW_CONF}"
        fi
    else
        printf 'Use existing %s\n' "${CONF}"
    fi
else
    printf 'Create %s\n' "${CONF}"
    cp "${REF_CONF}" "${CONF}"
    $PATCH_REF_CONF "${CONF}"
fi

# make sure there are uwsgi settings
update_conf ${FORCE_CONF_UPDATE} "${UWSGI_SETTINGS_PATH}" "/usr/local/searx/dockerfiles/uwsgi.ini" "patch_uwsgi_settings"

# make sure there are searx settings
update_conf "${FORCE_CONF_UPDATE}" "${SEARX_SETTINGS_PATH}" "/usr/local/searx/searx/settings.yml" "patch_searx_settings"

patch_uwsgi_settings() {
    CONF="$1"

    # Nothing
}

exec su-exec searx:searx uwsgi --master --http-socket "${BIND_ADDRESS}" "${UWSGI_SETTINGS_PATH}"

git init searx-nix

{
  inputs.nixpkgs.url = "github:NixOS/nixpkgs";

  outputs =
    {
      self,
      nixpkgs
    }:
    let
      supportedSystems = [ "x86_64-linux" ];
      forAllSystems' = nixpkgs.lib.genAttrs;
      forAllSystems = forAllSystems' supportedSystems;

      pkgsForSystem =
        system:
        import nixpkgs { inherit system; };
    in
      {
        packages = forAllSystems
          (system:
            let
              pkgs = pkgsForSystem system;
            in
              {
                default = pkgs.callPackage ./searx.nix {};
              }
          );
      };
}

{ lib, python3 }:

python3.pkgs.buildPythonApplication rec {
  pname = "luigi";
  version = "2.7.9";

  src = python3.pkgs.fetchPypi {
    inherit pname version;
    sha256 = "035w8gqql36zlan0xjrzz9j4lh9hs0qrsgnbyw07qs7lnkvbdv9x";
  };

  propagatedBuildInputs = with python3.pkgs; [ tornado python-daemon ];

  meta = with lib; {
    ...
  };
}

{
  lib,
  python3,
  fetchFromGitHub
}:
with lib;
let
  pname = "searx";
  version = "1.0.0";
in
python3.pkgs.buildPythonApplication {
  inherit pname version;

  src = fetchFromGitHub {
    rev = version;
    repo = pname;
    owner = pname;
    # If you update the version, you need to switch back to ~lib.fakeSha256~ and copy the new hash
    sha256 = "sha256-sIJ+QXwUdsRIpg6ffUS3ItQvrFy0kmtI8whaiR7qEz4="; # lib.fakeSha256;
  };

  postPatch = ''
    sed -i 's/==.*$//' requirements.txt
  '';

  # tests try to connect to network
  doCheck = false;

  pythonImportsCheck = [ "searx" ];

  # Since Python is weird, we need to put any dependencies we know of here
  # and not into ~buildInputs~ or ~nativeBuildInputs~ as one might expect.
  # As a starting point, just copy everything from ~requirements.txt~ and
  # hope for the best.
  propagatedBuildInputs = with python3.pkgs;
    [
      certifi
      babel
      flask-babel
      flask
      jinja2
      lxml
      pygments
      python-dateutil
      pyyaml
      # httpx[http2]
      httpx
      brotli
      # uvloop==0.16.0; python_version >= '3.7'
      # uvloop==0.14.0; python_version < '3.7'
      uvloop
      # httpx-socks[asyncio]
      httpx-socks
      langdetect
      setproctitle

      # sometimes the packages in ~requirements.txt~ may not be enough, so if something is missing, just add it
      requests
    ];

  meta = with lib; {
    # You'll fill this in later when upstreaming to nixpkgs
  };
}

# tests try to connect to network
doCheck = false;

pythonImportsCheck = [ "searx" ];

postPatch = ''
  sed -i 's/==.*$//' requirements.txt
'';

buildPythonPackage = makeOverridablePythonPackage (lib.makeOverridable (callPackage ../development/interpreters/python/mk-python-derivation.nix {
  inherit namePrefix;     # We want Python libraries to be named like e.g. "python3.6-${name}"
  inherit toPythonModule; # Libraries provide modules
}));

Ledger Nano S/X Both are great devices, except that the Nano S has this tiny flaw of not having enough memory to install multiple apps at the same time. It can be worked around by constantly installing and removing apps (you won't use your crypto), but that has a few disadvantages:

cumbersome - do you really want to shuffle apps around constantly? wears out flash - unnecessarily abusing the flash is a bad idea, especially since it's even less durable than normal flash some apps might not even fit - I can't verify this one, but I've read somewhere that some bigger apps might not even fit. And I don't mean like installing Bitcoin and Ada, but just installing Bitcoin by itself might not fit. That's bad

So after not-so-careful consideration I removed the Nano S from my list. Easy, one down, now's the final round.

Trezor T

Very good device, open source hardware and software, well supported. Some things I don't like are the form-factor, I prefer Ledger's USB stick like design, it stands out less.

version: "3.7"

services:
  nanos:
    build: .
    image: ledgerhq/speculos
    volumes:
      - ./apps:/speculos/apps
      ports:
        - "1234:1234" # gdb
        - "5000:5000" # api
        - "40000:40000" # apdu
        - "41000:41000" # vnc
        command: "--model nanos ./apps/btc.elf --sdk 2.0 --seed secret --display headless --apdu-port 40000 --vnc-port 41000"
        # Add `--vnc-password "<password>"` for macos users to use built-in vnc client.