secana

joined 11 months ago
 

Hi rustaceans! What are you working on this week? Did you discover something new, you want to share?

 

Kellnr, the crate registry for Rust got an update. The latest version contains bug fixes and dependency updates. If you are interested in hosting private crates that must not be on crates.io, check it out.

[โ€“] secana@programming.dev 6 points 1 month ago

An honorable goal ๐Ÿ˜€

[โ€“] secana@programming.dev 3 points 1 month ago (1 children)

Just released Kellnr (https://kellnr.io) 5.2.6 with some bug fixes. I really wish I had more time for new features but at the moment my spare time is rare. If you want to host your own crates on your own infrastructure, check it out.

 

Hi rustaceans! What are you working on this week? Did you discover something new, you want to share?

 

Hi rustaceans! What are you working on this week? Did you discover something new, you want to share?

[โ€“] secana@programming.dev 2 points 2 months ago

Thanks, I'll try that!

[โ€“] secana@programming.dev 2 points 2 months ago (2 children)

Sure. I import the certificates like this:

{ config, pkgs, inputs, ... }:
{
  security.pki.certificateFiles = [
    ./certificates/home.pem
  ];
}

where home.pem is a default PEM formatted certificate. It works fine to import the cert system wide this way.

If I enter the flake.nix and run a simple curl against the remote server I get the following, which is typical for a TLS certificate error.

curl https://webpage.home
curl: (35) OpenSSL/3.0.14: error:16000069:STORE routines::unregistered scheme

So it seems to me that the development shell does not pick up the certificates installed on the system. I can work around that by using an impure shell, but I think that this is not how nix should be used.

 

Hi!

I've ran into an issue with nix develop shells.

My setup:

  • Nix Darwin (macos)
  • Custom TLS certificates installed via nix darwin

Everything works as expected with the installed certificates, but as soon as I enter into a development shell with nix develop, the certificates are not available and thus, I get TLS errors that break whatever I'm doing in the dev shell. If I use an impure development shell, the issue disappears.

Is there a way to use pure nix develop shells which respect the installed certificates?

[โ€“] secana@programming.dev 2 points 3 months ago

I use that too. Unfortunately it does only work with Docker and on some machines I've only containerd+nerdctl available. Would be cool if test containers supports more than Docker as a runtime.

[โ€“] secana@programming.dev 4 points 3 months ago

I reworked the whole CI/CD pipeline for https://kellnr.io. Switched from Ubuntu as the base image to the official Rust (Debian) image. Additionally, musl targets are build and released on github. This should allow kellnr to run out-of-the-box on any Linux distro.

 

Hi rustaceans! What are you working on this week? Did you discover something new, you want to share?

[โ€“] secana@programming.dev 2 points 3 months ago (1 children)

Thanks for the response. You are right, the config was at the wrong path. Unfortunately, the config itself does not work, too. After a bit of testing around this worked for me:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: nix-cache-volume
spec:
  capacity:
    storage: 500Gi
  storageClassName: manual
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/mnt/k8s/nix-cache" # Needs exists before PV is created!
  persistentVolumeReclaimPolicy: Retain
***
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nix-cache-pvc
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: manual
  resources:
    requests:
      storage: 500Gi
***
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nix-cache
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nix-cache
  template:
    metadata:
      labels:
        app: nix-cache
        name: nix-cache
    spec:
      volumes:
        - name: nix-cache-storage
          persistentVolumeClaim:
            claimName: nix-cache-pvc
        - name: nix-cache-config
          configMap:
            name: nix-cache-config
      containers:
        - name: nix-cache
          image: nginx:1.27.0 
          ports:
            - containerPort: 80
          volumeMounts:
            - name: nix-cache-storage
              mountPath: /data
            - name: nix-cache-config
              mountPath: /etc/nginx/nginx.conf
              subPath: nginx.conf
              readOnly: true
          resources:
            limits:
              memory: "512Mi"
              cpu: "300m"
            requests:
              memory: "256Mi"
              cpu: "200m"
***
apiVersion: v1
kind: Service
metadata:
  name: nix-cache
spec:
  selector:
    app: nix-cache
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
***
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nix-cache-ingress
  annotations:
    traefik.ingress.kubernetes.io/router.tls: "true"
spec:
  rules:
    - host: "nix-cache.raspi.home"
      http:
        paths:
          - pathType: Prefix
            path: "/"
            backend:
              service:
                name: nix-cache
                port:
                  number: 80
  tls:
    - secretName: nix-cache-raspi-home-tls
      hosts:
        - "nix-cache.raspi.home"
***
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: nix-cache.raspi.home
spec:
  commonName: nix-cache.raspi.home
  dnsNames:
    - "nix-cache.raspi.home"
  secretName: nix-cache-raspi-home-tls
  issuerRef:
    name: ca-issuer
    kind: ClusterIssuer
***
apiVersion: v1
kind: ConfigMap
metadata:
  name: nix-cache-config
data:
  # See: https://www.channable.com/tech/setting-up-a-private-nix-cache-for-fun-and-profit
  nginx.conf: |
    events {
        worker_connections 1024;
    }
    http {
      proxy_cache_path /data/nginx/cache max_size=500G keys_zone=cache_zone:50m inactive=365d;
      proxy_cache cache_zone;
      proxy_cache_valid 200 365d;
      proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_504 http_403 http_404 http_429;
      proxy_ignore_headers X-Accel-Expires Expires Cache-Control Set-Cookie;
      proxy_cache_lock on;

      server {
          listen 80;

          server_name nix-cache.raspi.home;

          location /nix-cache-info {
              return 200 "StoreDir: /nix/store\nWantMassQuery: 1\nPriority: 41\n";
          }

          location / {
              proxy_set_header Host $proxy_host;
              proxy_pass https://cache.nixos.org;
          }
      }
    }

The config is an adaption from this blog post: https://www.channable.com/tech/setting-up-a-private-nix-cache-for-fun-and-profit

 

Hi! I would like to host a transparent proxy for cache.nixos.org on my local kubernetes cluster.

I took the following NGINX config https://nixos.wiki/wiki/FAQ/Private_Cache_Proxy and created all the folders on the mounted storage.

This is the kubernetes deployment:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: nix-cache-volume
spec:
  capacity:
    storage: 500Gi
  storageClassName: manual
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/mnt/k8s/nix-cache" # Needs exists before PV is created!
  persistentVolumeReclaimPolicy: Retain
***
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nix-cache-pvc
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: manual
  resources:
    requests:
      storage: 500Gi
***
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nix-cache
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nix-cache
  template:
    metadata:
      labels:
        app: nix-cache
        name: nix-cache
    spec:
      volumes:
        - name: nix-cache-storage
          persistentVolumeClaim:
            claimName: nix-cache-pvc
        - name: nix-cache-config
          configMap:
            name: nix-cache-config
      containers:
        - name: nix-cache
          image: nginx:1.27.0 
          ports:
            - containerPort: 80
          volumeMounts:
            - name: nix-cache-storage
              mountPath: /data
            - name: nix-cache-config
              mountPath: /etc/nginx/sites-available/default
          resources:
            limits:
              memory: "512Mi"
              cpu: "300m"
            requests:
              memory: "256Mi"
              cpu: "200m"
***
apiVersion: v1
kind: Service
metadata:
  name: nix-cache
spec:
  selector:
    app: nix-cache
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
***
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nix-cache-ingress
  annotations:
    traefik.ingress.kubernetes.io/router.tls: "true"
spec:
  rules:
    - host: "nix-cache.raspi.home"
      http:
        paths:
          - pathType: Prefix
            path: "/"
            backend:
              service:
                name: nix-cache
                port:
                  number: 80
  tls:
    - secretName: nix-cache-raspi-home-tls
      hosts:
        - "nix-cache.raspi.home"
***
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: nix-cache.raspi.home
spec:
  commonName: nix-cache.raspi.home
  dnsNames:
    - "nix-cache.raspi.home"
  secretName: nix-cache-raspi-home-tls
  issuerRef:
    name: ca-issuer
    kind: ClusterIssuer
***
apiVersion: v1
kind: ConfigMap
metadata:
  name: nix-cache-config
data:
  nginx.conf: |
    server {
      listen 80;
      server_name nix-cache.raspi.home;

      location ~ ^/nix-cache-info {
        proxy_store        on;
        proxy_store_access user:rw group:rw all:r;
        proxy_temp_path    /data/nginx/nix-cache-info/temp;
        root               /data/nginx/nix-cache-info/store;

        proxy_set_header Host "cache.nixos.org";
        proxy_pass https://cache.nixos.org;
      }

      location ~^/nar/.+$ {
        proxy_store        on;
        proxy_store_access user:rw group:rw all:r;
        proxy_temp_path    /data/nginx/nar/temp;
        root               /data/nginx/nar/store;

        proxy_set_header Host "cache.nixos.org";
        proxy_pass https://cache.nixos.org;
      }
    }

To use the cache I added it to the substituters.

  nix.settings.substituters = [
    "https://nix-cache.raspi.home/"
  ];

But when I try to use it, get the error:

# Trigger a download
nix develop nixpkgs#just
# Error message
warning: 'https://nix-cache.raspi.home' does not appear to be a binary cache

In the logs of the NGINX I see the following error:

2024/08/03 12:09:30 [error] 31#31: *3 open() "/usr/share/nginx/html/nix-cache-info" failed (2: No such file or directory), client: 10.42.2.7, server: localhost, request: "GET /nix-cache-info HTTP/1 โ”‚
โ”‚ 10.42.2.7 - - [03/Aug/2024:12:09:30 +0000] "GET /nix-cache-info HTTP/1.1" 404 153 "-" "curl/8.8.0 Nix/2.18.5" "10.42.2.1"                                                                             โ”‚
โ”‚ 10.42.2.7 - - [03/Aug/2024:12:09:30 +0000] "PUT /nix-cache-info HTTP/1.1" 405 157 "-" "curl/8.8.0 Nix/2.18.5" "10.42.2.1"    

Any ideas whats wrong? I'm neither an nix nor an nginx expert, so maybe it is something really simple but I cannot figure it out.

 

Hi rustaceans! What are you working on this week? Did you discover something new, you want to share?

[โ€“] secana@programming.dev 1 points 4 months ago

I'm not ML pro and never used Python or Rust for it, but I know that our ML team uses Python extensively for it. My gut feeling is that Python stays the king in the ML field but the underlying libraries are going to progress from C++ to Rust in the future. Or at least, if Rust gets stronger math/statistics libraries. If you get something cool running with Rust and ML, I'm interested to read about it.

[โ€“] secana@programming.dev 4 points 4 months ago (1 children)

Unfortunately not. But I try to work on it a few hours every week in my spare time. I think that having an easy and free crate registry is crucial for the adaption of Rust in the commercial space. Companies don't want to share their code publicly on crates.io. My full time job is in the IT security sector. My hope is that by pushing Rust as a safe language, we can close some fundamental design flaws that languages like C/C++ introduced and make software landscape more secure.

[โ€“] secana@programming.dev 10 points 4 months ago (3 children)

I ported the frontend for https://kellnr.io from vuex to pinia, which makes the code to hold state in the frontend much cleaner.

[โ€“] secana@programming.dev 3 points 4 months ago (2 children)

How is the overall ML story with Rust? Is it usable in comparison to Python?

 

Hi rustaceans! What are you working on this week? Did you discover something new, you want to share?

[โ€“] secana@programming.dev 1 points 4 months ago

The selling point of Ice (the underlying framework for libcosmic) is the cross-platform compatibility. Can I use libcosmic cross-plat as well, or is it more a specialisation of Ice for Linux with the clear focus on the Cosmic desktop? Would be cool to re-use some widget etc.

 

Hi rustaceans! What are you working on this week? Did you discover something new, you want to share?

 

Hi rustaceans! What are you working on this week? Did you discover something new, you want to share?

view more: next โ€บ