u/firestorm_v1

Work was tossing a NetApp FAS2720 with a pair of DS212 disk shelves. Each shelf has 10x 8TB hard drives and two 894G U.2 SSDs. I have all of the licenses necessary for the machines, so thankfully that's not an issue. I did have to wipe the storage and recreate it so Sec would be happy but being that I also have the license keys, it was easy to get it up and running.

The two controllers are connected together on e0a/e0b for controller-to-controller communications, and e0c/e0d go to the 10G core switch in a lacp bond.

The target host is a Proxmox host with dual 10G nics in a bond, connected to the same core switch. For giggles, I did a basic test to write 50G of random data to an LVM volume created on the NetApp mount and on another iSCSI filer and found that the two filers didn't really differ much in time.

For the test, I used this command: time dd if=/dev/urandom of=./testfile bs=1G count=50 conv=fdatasync

In both test scenarios, a 100G LVM partition was created from the respective filer's volume group and prepared using EXT4. The filesystem was mounted, then the above command was ran.

From the Proxmox host to the other filer, it took 4m15s to write the 50G test file.
From the Proxmox host to the NetApp filer, it took 3m53s to write the 50G test file.

While I do have the SSDs, I do not think/remember if they were incorporated into the iscsi volume. When I look through NetApp's documentation for the FAS2720, the only thing I see is 'FlexCache' but from what I understand, this is only for a local cache against a remote volume, not a local volume.

I don't expect flash-based speeds, but I figured that the NetApp with custom hardware designed for storage would handily beat out a NAS built on commodity hardware that has no benefit of customized hardware. Looking at the dashboard, it seems that the NetApp barely breaks above 500IOPS. Latency hovers around 0.17ms, and throughput is usually 1.34MB/sec.

Any suggestions on how I can improve performance on the FAS2720?

I was fortunate to be able to get my hands on a few SRX300's for labs and training from work, but I found quickly that the OS installations on them were ancient, and one of them has a bad eUSB flash module.

Through combinations of TFTP, HTTP, and dumb luck, I've managed to get the eUSB module replaced with a generic 32G USB stick and I find myself at the great bridge between worlds (architectures).

I've got the machine running junos-srxsme-24.2R2-S5.3.tgz, and following Juniper's documentation, I should be able to install the mips64-24.4R2-S4.12 and have had zero luck. Running 'request system software add (filename) no-validate' goes through the motions of the initial staging and reboot. The SRX reboots as normal, goes through both 'chances' to interrupt the boot process, then it stops at "Installing Junos OS release 24.4R2-S4.12" and there it sits. I've waited several hours because I know this is a fundamental architecture change but it never proceeds past that point. I end up having to remove power and reconnect to get it back into a usable state, then load the "last known good" OS via loader so I can get the SRX to respond.

- I've tried coming in from 23.4R2-S3 and 24.2R2 versions to mips-64-24.4R1.9 and mips-64-24.4R2.21 but both target versions exhibit the same symptoms.

- I've tried getting into loader, then using boot -v for verbose output. Everything else has verbose output when booted in this mode but it all stops at "Installing Junos OS release".

- I've tried the usb autoinstall..conf trick on another USB device to get it to work but same issue. It stages but when it reboots, as soon as it gets to the "Installing Junos OS release" it freezes.

- I've tried mounting the partitions using singleuser mode to see if I could find any logs that would indicate what the problem is.

- I've tried verifying that I have space available on the partitions.

- I've tried extracting and updating U-Boot, and compared versions on machine with versions in the target version and it matches every time.

- I've tried examining the filesystem layouts and have confirmed that it matches the "new" partitioning for v24.4+

At this point, I'm beginning to think it's a lost cause and this 300 will be stuck on 24.2R2 versions..

Since these devices have been out of production for so long, I doubt they're covered under any support contract however because we do have other active supported devices I am able to get the files for updates.

I don't care about the data on the drive, we didn't use any of the license features and these devices apparently were factory reset before being put in to storage so there's no userdata (certs/keys/configs) to save. I am completely fine with wiping the onboard storage key if it means I can get this machine over the bridge. I want to see if I can get this one over the architecture hump so I can repeat this a couple more times upgrading the other two.

These are the last lines on the console when it freezes. No amount of waiting will change this.

Installation of disk:/upgrade/install.tar
** /dev/da1s3f
** Last Mounted on /cf/var
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
422 files, 787591 used, 12417906 free (282 frags, 1552203 blocks, 0.0% fragmentation)

***** FILE SYSTEM IS CLEAN *****
Setting sane date:
Sun Aug  2 09:36:00 UTC 2026
Installing Junos OS release 24.4R2-S4.12 ...

Any suggestions?

After reading through many of these posts, I can't be the only one that has a relatively sane (so far) Claude experience?

Some of the content I've read (here and others) sound like Claude is bringing about the End Times, is demonically possessed, or has gone clinically insane.

My setup isn't fancy, it's just the Claude application on Windows, with Co-work on and restricted to a single directory where my code lives. If anything Claude has been super helpful with a handful of projects like refactoring my OpenVox codebase, hiding and retrieving discovered secrets using encrypted YAML and so on. It's helped me confirm or refute several suggestions on hardware buildouts when the OEM wasn't really clear on their documentation. and it's currently assisting me through a rather precarious firmware upgrade that's quite stubborn.

If it's anything, it's been too sane. I keep having to tell it "Yes, it's OK to read github." for the millionth time, or "Yes it's Ok to read (this vendor's site)", "Yes you can edit files in the Cowork directory path), etc..

I'm curious as to what y'all are running that could be attributing to Claude's insanity. It's been an interesting read to find out about when Claude goes wrong (my condolences to the person that had their NVME wiped by Claude). Anyone having Claude trying to translate the Necronomicon or something?

I got to thinking about datacenter deployments and some of our non-Proxmox nodes use iscsi for both OS storage and for the volume storage for the VMs they host. With our other deployments, the boot iscsi volume has the OS for the machine, and the boot volume has the necessary configuration to mount the data/storage partition.

Has anyone tried using iscsi boot to boot a node to Proxmox then have it connect using a LVM to iSCSI storage backend for VM storage?

This is more of a 'Could this work?' versus an actual deployment model, I just thought it would be neat to try it and wondered if anyone else had done this.

I seem to find myself up an estuary of effluent without a means of locomotion and I'm hoping someone here can help save this box.

First off, this box is _ancient_, when I started, it was on Debian Squeeze. I've managed to successfully upgrade it over and over through Wheezy, Jessie, Stretch, Buster, Bullseye, Bookworm, and it's been working just fine with the usual upgrade path (change sources.list, apt update, apt dist-upgrade, apt autoremove, reboot).

I decided to upgrade it to Trixie using the exact same process and as I was watching apt do its thing, I noticed it started removing various packages that were really important, then it asks about removing the kernel which was unusual. Thinking apt knew what it was doing, I stupidly answered 'Yes' and apt then kept on going.

Before proceeding with the reboot, I decided to check to see if there is a kernel installed, and sure enough, there's not! Ok, well that's a new one.

apt install linux-image-amd64 yields dependency issues for kmod and libc6 and "Unable to correct problems, you have held broken packages"... That's odd, I don't have any packages held according to apt-mark showhold.

root@littleblackbox-vm:/etc/apt/sources.list.d# apt install linux-image-amd64
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 kmod : Depends: libkmod2 (= 34.2-2) but 30+20221128-1 is to be installed
 libc6 : Breaks: sysvinit (< 3.09-2~) but 2.88dsf-41+deb7u1 is to be installed
E: Unable to correct problems, you have held broken packages.
root@littleblackbox-vm:/etc/apt/sources.list.d# apt-mark showhold
root@littleblackbox-vm:/etc/apt/sources.list.d#

I tried doing apt-get dist-upgrade, but it just exits cleanly while listing a bunch of packages that have been kept back despite nothing in the configuration saying that they've been kept back.

I was able to get a kernel installed by changing sources.list to point back to Bookworm but trying to upgrade to Trixie just starts with more dependency hell:

oot@littleblackbox-vm:/etc/apt# apt dist-upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Error!
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 initscripts : Breaks: kmod (< 34.1-1) but 30+20221128-1 is to be installed
               Breaks: util-linux-extra (< 2.39.2-2.1~) but 2.38.1-5+deb12u3 is to be installed
E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.

I'm not using any custom repos or anything out of the ordinary, it's just a basic webserver. I'm more caught off guard by the fact that I've managed to take this box from ancient to relatively recent but now on the last upgrade to get it to 'current', it decides to explode in such a horrid fashion.

Any ideas how I can get this upgrade completed?

EDIT: Well that was a fun trip down memory lane. Ends up that sysvinit was the cause of the problem and the solution. All I had to do was remove sysvinit forcefully and restart the upgrade. It took a couple of tries to find what all was missing and I had a brief minor panic attack when GRUB couldn't find the kernel but that was because update-grub was missing. I was able to get the box running again, installed a few more systemd units, some missing apt packages (like openssh and sudo) and got GRUB updated. Now I'm off to the races, apache2 is happy, openssh is listening as expected, I can sudo and get to roor, networking works, and all is well.

Thank you all for helping me drag this box kicking and screaming into a current version!

When our fleet of three Litter Robots work, they are an absolute godsend, however when they fail, it's usually for the stupidest of reasons. For example, the latest issue appears to be with our newest (newest being relative, it's about two years old now) robot Vincent and its godforsaken pinch sensor.

Of course, it couldn't start happening when I'm at home and can easily reset the box before bed, instead it waits until I'm in the middle of the Atlantic Ocean, about the worst possible place to be to need to return the machine to working order. This has lead us to question the dependability of the LR3's pinch sensor as a while. Our other LR3's are not immune either, they will randomly throw a pinch sensor fault, but generally resetting the box and allowing it to cycle again fixes them right up.

I'm tired of kicking the can down the road for the next three months and I'm definitely tired of having the boxes be unreliable when we're on vacation. I'm looking for a long term, reliable solution so I don't break out in a cold sweat every time I see the icon of the Whisker app in my notification bar.

Has anyone found a working solution for this rather frustrating issue? Any luck with lithium grease, dielectric grease, or petroleum jelly to protect the contact surfaces? Any ideas on any hardier switches versus "two strips of metal held in place with plastic bits" that I could consider replacing the pinch "sensor" with?

The first two robots have had their DFI assembly and pinch sensors replaced with the version 2 sensors that fixed the corrosion on the DFI circuit board. Vincent shipped with the version 2 sensors but still had the same strips of metal for the pinch sensor that the version 1 DFI came with. Is there a metallurgical difference between the v2 and v1 pinch sensor metals that would explain why the older LR3's with the replaced DFI assembly would be more reliable than the newer LR3 that shipped with the newer DFI assembly?

In my continuing quest to see if I can revive the screwed up AFF-A300 controller in my filer at work, unfortunately I find myself not familiar enough with the low level hardware and hope someone can lend a hand and let me know how it should work. At last attempt, the AFF-A300 secondary controller would not power on using SP commands, and unfortunately still does not. The good news is this filer is now removed from production and the last tenant has been removed from it giving me a lot more latitude to mess around and see if I can cajole it into working again.

At one point, I had halted the working controller and powered down the system, then disconnected all cables except for power, e0m management, and the serial console ports on the two controllers. When I powered up the controller chassis, I got no serial output. Hitting Ctrl-G, I was prompted to login to the service processor on both controllers and was able to access both, however neither service processor will power on. Both controllers have the amber LED lit (second from top), but neither have the green LED lit (unlike a fully working NetApp). To make matters worse, I can only power on the first service processor (at least "system power on; system power status" returns "Host power on"). The second service processor remains at "Host power off" regardless of how many times "system power on" is issued with no error in logging.

Finally, to my question. With all of the SFF8644 cables disconnected as well as all the 10G NICs (including the cross-cluster cables), will the AFF-A300 service processors still attempt to "boot"? Should I see BIOS messages on the serial console when power is applied to the chassis, or does it see that there's no disk shelf attached and just give up with no error displayed?

I'm at the point now where the boss wants to know if we should gamble on buying another service processor and try and switch the serials on it to get the filer working again, or just cut our losses, ewaste the AFF-A300 and move the storage shelf to another AFF-A300 chassis that is still in production. Without knowing the hardware as well as I'd like, I can't say authoritatively either way as I don't know if I've unintentionally lobotimized the only working service processor in the filer by disconnecting all the storage, or if it should be attempting to boot off the built-in M.2 storage in the service processor itself.