u/Apachez

I have a thing of attempting to make my life easier by using immutable OS as host for containers.

Such option is VyOS which originally is a software based router but its great to host containers aswell.

That is I install the ISO and use a single config file to configure the whole system. Compared to solutions such as Thalos I wont need any additional mgmt-clients in order to be able to configure the installation.

Now Kasm single-server already exists as a VM-image through Kasm-1.19.0-Single-Server.ova

But downside for me is that above is using Ubuntu 24.04 LTS (speaking of which, when will there be a 26.04 LTS release?).

That is compared to VyOS it will take alot more effort to have this updated specially in environments where there is no Internetaccess. Sure you can setup and maintain something like Proxmox Offline Mirror but that will still take alot more effort than just download and transfer the ISO when VyOS gets a new release.

During initial investigation of "how hard can it be" to make Kasm to properly run on VyOS I think there might be some speedbumps along the road:

1. Will Kasm work with Podman instead of Docker or are the Kasm containers hardwired to use "Docker" as binary (for the automation of starting and stopping the workspaces upon request etc)?

2. I have seen that the install script is draggnig along a networking and a logging plugin - how critical are these for Kasm to be working and if not critical then what will I be missing without them?

3. How likely is it that I will make Kasm work on VyOS as host who uses Podman instead of Docker (and practically doesnt allow for install of network or logging plugins since the use of overlay filesystem would mean that these will be gone when you update the installation to next release)?

4. Are there some dummy instructions on how to manually install Kasm using containers (other than trying to manually decipher what the install.sh is doing)?

Obviously these containers are needed for core functionality:

kasmweb/agent:1.19.0-rolling   
kasmweb/api:1.19.0-rolling   
kasmweb/kasm-guac:1.19.0-rolling   
kasmweb/manager:1.19.0-rolling   
kasmweb/postgres:1.19.0-rolling   
kasmweb/proxy:1.19.0-rolling   
kasmweb/rdp-gateway:1.19.0-rolling   
kasmweb/rdp-https-gateway:1.19.0-rolling

And then ontop of these comes whatever workspaces you wish to be able to run.

I am currently using: Kasm-1.19.0-Single-Server.ova

Is this command safe to be used along with Kasm?

> sudo docker system prune

Since I assume adding "-a --volumes" would be a bad idea since images should exist but are not necessary currently used by a container?

Are there other clean up commands that can be good to run on a Kasm-server?

Im thinking when using persistent directories and then you have deleted one or more users, do there exist some automatic way to swipe away directories who belongs to a user who no longer exists or is that up to me as an admin to go on a hunt for to clear up "unused" diskspace?

How are images kept up2date - does this occur on every start as a container or are there some other black magic being involved?

Like if I wish to do the updating on a fixed schedule (during allocated maintenance window)?

I assume its something like "2GB (2048MB) should be enough as default for everybody" but from where comes the additional 720MB?

With Proxmox for example who uses QEMU/KVM when you assign lets say 2GB (2048MB) of RAM for a VM-guest then there will be additional lets say 256MB if you also use virtio-GPU as virtual display which will be used like a framebuffer by the VM-guest. Otherwise the standard console uses something like 16MB or so.

Is it something with Kasm workspaces aswell and how does this magic formula looks like?

If I want to adjust this so the workspace can use 4GB (4096MB) of RAM - should I then type it in as 4816 (4096 + 720)?

Im currently using: Kasm-1.19.0-Single-Server.ova

When adding a workspace they all seems to default to cores: 2 and memory: 2768.

And I can in Admin -> Workspaces -> Workspaces click on Filters and select that both cores and memory columns should be seen (will save on clicks to see these settings in a table without having to first click on edit each and every workspace).

But it still annoys me that added workspaces always seems to use cores: 2 and memory: 2768 where I need to manually adjust that to cores: 4.

It feels like I have been looking all over the place but is there no global setting somewhere that will automatically assign number of cores for added workspaces?

Or are these numbers strictly from the packaging of the workspace itself so Im stuck with manually hunt down and change workspaces from 2 to 4 cores?

Im probably missing something here...

I am currently testing Kasm-1.19.0-Single-Server.ova and that works like expected.

So I thought I would give it a go to add another workspace.

So I tried a "sudo docker pull" as below which worked without issues but then when I run "sudo docker images" the newly downloaded image is nowhere to be found?

kasm_admin@kasm-server:~$ sudo docker pull docker.io/kasmweb/audacity:1.19.0-rolling-daily
1.19.0-rolling-daily: Pulling from kasmweb/audacity
4f4fb700ef54: Pull complete 
4d49e951ec3b: Pull complete 
c36cc4ab8e70: Pull complete 
aaca8373ae59: Pull complete 
0f40056b92eb: Pull complete 
ea76751773ea: Pull complete 
3baba0fdd223: Pull complete 
801ec770692d: Pull complete 
ebc8d8e5c003: Pull complete 
Digest: sha256:3528c27eba4b4c01c23581c652382ce0e84720c901939a004a7ac434dcb7f6c1
Status: Downloaded newer image for kasmweb/audacity:1.19.0-rolling-daily
docker.io/kasmweb/audacity:1.19.0-rolling-daily
kasm_admin@kasm-server:~$ docker images
permission denied while trying to connect to the docker API at unix:///var/run/docker.sock
kasm_admin@kasm-server:~$ sudo docker images
                                                                                                                                                                                      i Info →   U  In Use
IMAGE                                      ID             DISK USAGE   CONTENT SIZE   EXTRA
kasmweb/agent:1.19.0-rolling               eb8e8ff4014a       1.21GB          336MB    U   
kasmweb/api:1.19.0-rolling                 8a4f8d92aaa8       1.33GB          232MB    U   
kasmweb/chrome:1.19.0                      25389cc8eafa       4.75GB         1.41GB        
kasmweb/firefox:1.19.0                     0ae54ed42da4       4.55GB         1.33GB        
kasmweb/kasm-guac:1.19.0-rolling           1a0e82e3bb6b       1.24GB          335MB    U   
kasmweb/manager:1.19.0-rolling             f52d2424b713       1.33GB          232MB    U   
kasmweb/postgres:1.19.0-rolling            be2d40095a72        440MB          118MB    U   
kasmweb/proxy:1.19.0-rolling               0c8a003958cd        206MB         64.7MB    U   
kasmweb/rdp-gateway:1.19.0-rolling         4b42a2f56c81        618MB          155MB    U   
kasmweb/rdp-https-gateway:1.19.0-rolling   9d93c4c7cda9       95.8MB         32.3MB    U   
kasmweb/terminal:1.19.0                    4403ce7d6980       4.83GB         1.32GB        
kasmweb/ubuntu-jammy-desktop:1.19.0        fba645c461a3       14.9GB          4.1GB        

So what am I doing wrong? :-)

Looking through the manual there seems to exist these options:

https://www.kasmweb.com/docs/latest/guide/storage_mappings.html

https://www.kasmweb.com/docs/develop/guide/persistent_data/volume_mapping.html

But none of them seems to be doing what Im looking for.

Im basically looking for the way you can have a shared folder when using RDP through the RDP-session itself.

The usecase is when using Kasm container as a jumpserver and you need to for example update the firmware of a destination device (client <-> Kasm container <-> destination device) or the reverse when you got some backups (like config and whatelse) from a destination device and wish to in an easy fashion get these files to your client.

When comparing to KVM-over-IP solutions such as GL.iNet Comet and similar you often here have a way to upload/download the files to the KVM itself through the browser which then exposes this directory to the destination (basically will show up as a USB-drive at the destination). This way I can import/export files to the remotely managed device and from there upload/download files to the destination device.

Does something similar exist with Kasm or is in the todo-list/pipeline?

What are your protips regarding optimizations for VyOS when being runned as VM-guest?

Something like:

set system option performance 'virtual-guest'

seems obvious but what else?

Would this considered be good or bad aswell?

set system option kernel cpu disable-nmi-watchdog
set system option kernel disable-hpet
set system option kernel disable-mce
set system option kernel disable-softlockup

And what about offloading per interface?

Should all offloading be turned off when runned as VM-guest or are there any offloading that still helps?

And what about firewall rules?

I will be using services local to the VyOS for my clients (NTP, DNS etc).

So in terms of filtering I would be happy with "screening router" aka regular ACL if compared to L2/L3-switch.

Would firewall rules with notrack be sufficient and how should they be written or will flowtables work here at all?

Differences between KVM (QEMU / Proxmox) and VMware or for that matter XCP-NG?

Im mainly thinking if someone perhaps already benchmarked this and can share your results?

Looking at https://hub.docker.com/r/technitium/dns-server the oldest version that exists seems to be 14.0.0.

Dunno if thats the first container edition or not but anyway.

If I would start with 14.0.0 can I then just reload into 15.2.0 and it will just work (regarding config-files) or how does upgrading works?

Is it also possible to downgrade?

Like if I got a 15.2.0 installation and replace the image with an older one - how far back can I moonwalk?

Im thinking since its not uncommon (compared to others) that if you for example would have been on version 13.0.0 you must upgrade through all major version like from 13.0.0 to 14.0.0 then 15.0.0 and finally you can jump onto the last one currently being 15.2.0.

And also that downgrading outside of current majorversion is often not supported.

Like if I currently have 15.2.0 I can downgrade down to 15.0.0 but not like back to 14.x.x or below.

So whats the official support for upgrading and downgrading containers with Technitium?

And what have those of you running technitium/dns-server experienced in the wild regarding upgrading or downgrading?

My main concern is how critical it is to apply each update.

Unless there are some CVE findings or some other bugs/features affecting me I would normally no jump onto each released version (unless I got some spare time to call for another maintenance window). But at the same time waiting for too long, at least with others, will also cause a headache.

Im trying to do some housekeeping of which images is used and instead of relying on tag:latest I try to figure out what is the actual latest version and fetch that tag.

This way I can also better track of what is actually being runned where.

So when I take backup of an image it will be stored as (for example) technitium_dns-server_15.2.0_260624.tar.gz

Any of you who happens to know if this is possible purely through CLI?

You can use "sudo podman inspect" to find out the digest of an image.

But I assume you then cant pull that with its current tag from a repo such as docker.io?

Currently Im doing a manual method of visiting https://hub.docker.com/r/technitium/dns-server and in the dropdown for Recent tags compare their digest.

Using the above manual method I found that for https://hub.docker.com/r/technitium/dns-server the digest between tag:latest and tag:15.2.0 is different - how come?

Is tag:latest newer than tag:15.2.0 when it comes to technitium/dns-server?

I have a host setup with currently two vrfs like so:

VRF MGMT: ETH0: 10.1.0.1/24

VRF PROD: ETH1: 192.0.2.1/24

When I use the container with host-network enabled I can make it listen at each interface so that webgui of the container (used for management) can listen on 10.1.0.1 and the production traffic (being dns in this case) can listen on 192.0.2.1.

For TCP-traffic this works without issues but UDP-traffic fails.

It seems like the container isnt properly using srcip/srcinterface for UDP-replies so UDP-packets arrives to the container but never return back to the client.

Now if I use the same container without vrfs at the host then both TCP- and UDP-traffic will work.

What are my options (I have filed this as a bugreport towards the one who made the containter) as workaround natively in docker (other than disable vrfs at the host)?

How have others resolved similar issues?

I was thinking of setting up a custom network in docker for this purpose and then somehow at the host portmap from each vrf (and interface) to whatever port will be exposed in the container.

But that didnt work at first attempt so either this will not work at all or Im doing something wrong?