u/CeC-P

All of a sudden this windows 10 computer (yeah, yeah) that just needs to VPN into an offsite server to run a time clock app started failing to do so. It connects but after you connect, you can technically log straight into the firewall by IP, so I know the connection worked, but I can't get any DNS to load, no websites, and can't ping 8.8.8.8 for example. As far as we know, nothing changed.

So I exported the profile, installed Forticlient 7.4 on a brand new Windows 11 25h2 virtual machine at our office, which is a different IP and ISP, and it connected fine but also killed all ability to load websites, etc. We don't think anything changed on the firewall and it reports healthy so not sure what could cause this all of a sudden. And automatic firmware updates are actually disabled on the firewall (so I'm patching it to 7.10 then 7.13 sequentially tonight off-hours).

Anyone see this weirdly specific issue?

https://preview.redd.it/xkvxbqi55gah1.png?width=728&format=png&auto=webp&s=3a12b54121125d4bebcc8a1c846dcd15355801ea

I assume they were trying to hit the "this will explode in pyrotechnic hell-blast if it's damaged" square directly but missed a little to the right.
Btw my cousin worked for a delivery company and said 1000% the a-holes loading the trucks are behind this.

I work for an MSP and we just deployed about 5 new printers this year. The customer has now destroyed 2 of them with Amazon counterfeit cartridges and one is refusing to "connect" to the chips and refusing to print on 3 of 4. So now they're out the money for the counterfeit garbage, the money for 2 printers that the carts exploded in, and we have to drive out and attempt to repair/clean one after they get real cartridges for it and hope the system that puts the toner onto the drum isn't damaged by badly out of spec molecules, which is what it sounds like.

Good thing they saved so much money shopping on Amazon for "just as good" cartridges. American medical care provider btw. That should scare you.

There is no right answer here but I'm wondering what the strategy is for you guys. I've had CIOs say they want all the same model laptop/desktop so they can stop tracking down individual problems with specific hardware. Obviously that's incorrect and I can assure you that it'd have been easier to flash 20 USB C power delivery chip firmwares instead of 100 when users are in the field. And then the same again with defective Ryzen drivers that kept crashing our remote control. And we have to fix it on remote control. Also, I can track down specific hardware defects after never seeing the same laptop twice at my non-commercial computer repair store for 8 years. But I believe my skill set is unique in the corporate IT world.

Obviously imaging efficiency is a factor. I just don't like putting all my eggs in one basket, however, if I'm charge of purchasing then the best laptop for the price and performance is very obvious. Going one step down is unacceptable and one step up is a waste of money on the order of $100+ typically. I am VERY sick of diagnosing specific motherboards' random incompatibilities with DP passthrough on random models and specific firmwares of docks though. If we had all the same model, we'd have simply fixed it, everything is flashed to a version that works, and there's a lot less tickets.

I think right now you just buy either nothing or what's in stock for the cheapest because Sam Altman ruined the entire world so perhaps describe your strategy before that, and if it backfired spectacularly or worked great.

Just kidding. MSP here and it turns out it was actually the fact that the two main switches are under the secretary's desk, because duh, where else would you put them? And she runs a space heater if it's below 85F in there. Turns out snagless CAT6 housing is also known as heat shrink tubing and it will squeeze the plug and eject the Ethernet cable on its own, if hot enough for long enough.

Yes, we have told her it's not ideal to do that. No, she doesn't care.

I picked the wrong week to stop sniffing glue.

This is all the data I could compile on the problem. I had to do heavy research on it because I never looked into or cared how any of this key and cert and boot time stuff worked until it affected my job directly. It was just monopoly abuse against Linux in my opinion. I'm no expert on this so if I get anything wrong, leave a comment, and I'll edit this post.

What happens if you do nothing

June 24th - absolutely nothing happens. Any system running a modern Windows OS will work because it was loaded with the old Secure Boot allow list and the boot loader is on it, obviously, or it wouldn't have been able to boot.

Exceptions: none that I could find

Long term if you do nothing problem #1
You try to install Windows 11 26h2 onto a computer that never had a BIOS firmware update with the list of new allowed certs/private keys/whatever. It fails to boot from the flash drive installing it, for example, because the new boot loader isn't on the allow list.

Solutions:
- turn off Secure Boot and install it anyway. Windows currently does not require it in order to install. Very little software needs secure boot to be on in order to run. The only ones I'm aware of are anti-cheat systems from EA, Activision, and Blizzard. OH NO. But the following may throw errors and/or not work in Windows:
Memory Integrity (HVCI)
Device Health Attestation tests
Windows Defender System Guard
Secured-core PC features
Enterprise compliance policies in general

Install the OS then turn on Secure Boot after the OS is installed.

Note: If you turn on Bitlocker then flash the BIOS from within Windows then turn on secure boot in that exact order, it will ask for the recovery key at next boot. Leave Bitlocker off until the end of the process if this is your plan.

- boot to a windows PE like Bart PE and then patch the bios with whatever simple standalone patcher thing Dell/HP/Lenovo/whatever provided. Then install the OS after that.

- Don't load an OS and first flash the BIOS from within the BIOS using a usb drive containing the new ROM. This is more commonly a capability of non-OEM MSI, ASUS, gigabyte motherboards, not corporate OEM hardware.

Long term if you do nothing problem #2
MS finds a vulnerability in the old boot loader/cert system/encryption method and adds it to the revoke list. Maybe quantum computers crack it or something. They push out the revoke list via windows update. Now any system without BOTH the new BIOS firmware and new patches on the Windows OS side instantly overnight won't boot until you turn Secure Boot off in the BIOS config, causing a near Crowdstrike level event.

IN MY OPINION, this is not a realistic scenario.
- how did a system that's stuck on 23h2 for example receive a windows update at all after end of life?
- why did the windows update that revoked the old boot loader run but not the much earlier one patching it run?
- okay, a computer has windows updates turned on but filtered by a 3rd party patch management system at your company, it can't patch to a later build of windows because there's not enough space on the SSD, but all other updates are working. You don't run Dell Command (for example) to patch the BIOS because BIOS patches are scary and then you blocked the same firmware patch pushed out from the OEM via windows update in the critical drivers channel. Then you accidentally let through the theoretical future insecure boot loader revocation update. In that case, you're a terrible sysadmin. You get to now get off your ass and go turn off Secure Boot one a time, per affected computer so they can boot and then clean up the problem later.

But realistically, a tiny, specific secure flaw that still probably needs a UAC bypass to run anyway is found so Microsoft's solution is to release a patch for OS versions that isn't supported/getting updates anymore and the sole and entire purpose of the patch is to make the OS not bootable because the OSes own, current boot loader is now on the revoke list. The next day the media says Crowdstrike 2.0, everything is down, it's Y2K, OMG, except MS knowingly did it on purpose. Then accusations fly about it being planned obsolescence and the feds throw the book at them. Why are they even patching a system after saying "we're no longer providing patches for this build of Windows." I don't think the terms and conditions you signed, or arbitration limitations would prevent MS from getting sued out of existence if they purposely, knowingly released an update that destroyed older computers "for security."

Realistic side effects:
All your old bootable utilities like PEs or Macrium (for backup and image deployment) stop working.

Solution: get new ones or turn off Secure Boot as needed (which breaks bitlocker). Yay, more budget expenditure for Macrium 2027 edition or more IT staff time. Oh well. It's still cheaper than RAM.

SCCM and/or Autopilot breaks
It already works like crap on any given day, especially together. Add another problem to the pile. Okay, okay, I know next to nothing about those systems and how they work or how they relate to Secure Boot. One company I worked at used both, they work like crap, and MS keeps changing them. Oh well, good luck. You should have pushed out BIOS updates before June 24th then and maybe reconsider using magical MS tools that do everything perfectly and make your life easier because they never do.

Solution: (my best guess) Patch the BIOS manually via a working, modern PE. It takes about 3 minutes if you're in front of it. If your users are remote, then that sucks. Or use the backup method for imaging and deploying new computers that you already have in place for when either SCCM or Autopilot fails. If you don't already have one, you do not live in the real world.

Outlyers:

1. Extremely custom boxed hardware/software solutions running windows with a very custom motherboard with kiosked OS environments that aren't supported by the manufacturer anymore, so no BIOS updates to get the new allow/revoke list for secure boot loaders. People are throwing a fit over these since you typically can't turn off Secure Boot in their custom BIOS configs or you don't even have the BIOS password in the first place.
   - well they either are or they aren't getting Windows updates if they're highly controlled, custom solutions made by 1 vendor. So how would they not get the new Windows updates that fix the problem but do get the Windows update that revokes boot loaders? Block patching for them completely (many methods for this) until you can replace the systems or just live with the horribly insecure, locked down, canned garbage that you shouldn't have bought in the first place.

2. You had a laptop brand new in the box made in 2023 and you deploy it in 2027 and image deployment fails because the installer uses a newer boot loader.

Solution: WTF is wrong with your company? Ever heard of depreciation and CMOS battery failure? Then update the BIOS manually before deploying the new OS via the methods outlined above. If fact, you don't even need a PE. Just boot to the OS it came with, patch the BIOS, then pave over it with the image deployment method you already use.

3. Anything listed already except the users, the computers, etc are 100% remote and management thought it would just sort of work itself out somehow magically and nothing major would ever happen.
   Solution: Yeah, that is a problem. You probably should have planned for that, as 100% remote IT management inherently doesn't work. Oh well, hope you have a good rate with a shipping carrier. Management knew what they were getting into with that business model.

4. Something goes wrong with the patching chain and it somehow causes Bitlocker to trip in large numbers.
   Solution: that already happened.

5. Your company uses custom Secure Boot keys
   Solution: why the **** did you think that was a good idea? Time to find a new job!

6. The hardware is 10+ years old and the makers never made a BIOS patch to work with the new certs/boot loaders.
   Solution: Yeah, it wasn't going to run forever. Either a failed hardware component or other security vulnerability was going to take it down eventually. It's probably susceptible to rowhammer/specter/meltdown/etc right now anyway. You should have been looking into replacing it, despite difficulty and cost, already. Now you're being forced to do so. Or block that Windows update on it or reinstall build 25h2 or go turn off Secure Boot in the BIOS by hand, in person, before June 24th.

Hopefully I didn't miss any fringe stuff or important details but let me know and I'll edit the post. 0% of this was written by AI and the whole post is public domain licensed.

Setting up 5 brand new Yealink T54W phones with Ring Central. I provisioned them so they're all in the admin portal on the web. They updated the firmware as part of that. Then I noticed one of those two things reset the admin password to default. Now it's also complaining that user "user" and user "var" are using default passwords.

I assume user will be filled in when I assign a user/extension to the phone. But should I change var, as that's an internal type of user. Also, should I then change the admin user account on the phone via the web interface of the phone itself on our local network, or will that somehow lock Ring Central out?

Also, can I change any of these centrally from Ring Central's admin page? It'd be a lot faster if I could.

I just found out they technically do exist and some of them are even not scams/counterfeits. I was tracking some down for use in a small battery pack that will have VERY high cycles. Then a source told me that 18650 LiFePO4 cells tend to be rated for 3.2V and top off at 3.65V.

That means they won't work in a flashlight or traditional battery back with USB charge. But that's fine, I was going to put them in a small 12V arrangement. But then they'd have to be 4SxP instead of 3SxP arrangement to get to over 12V. That's kinda weird. I have to check the input range on my intended inverter because 14.4 is pushing it for some. Anyone else use these ever and anything else I should watch out for?

Remember the late 1990's when people would steal 128MB sticks of pre-DDR RAM worth about $300 each from computers before resigning or getting fired so they put padlock loops on the desktop cases? Yeah, they're like $400 a stick now for 64GB setups. We had a request to do so by one of our MSP customers after we can't really prove it but we're 99% sure someone stole a stick.

Considering I can get past a dollar store bulk padlock that small with a paperclip, I instead put in an RMM rule that says send a high priority alert email if the RAM on a system falls below what it is now by more than 10%. I had to hard code it since that wasn't a trigger template for some reason.

Anyone else already run into this and doing something similar? For everyone else, not a bad idea.

It has been upgraded from debacle to train wreck now, but we picked up all the pieces of the train strewn about and are good to go now, after it got so much worse!

This is too great not to follow up on. Remember the "I need to disable a stolen laptop without destroying any data or accounts but net user active:no won't work because it's a domain account" post?

Short version: we're an MSP. A company was shutting down. There was a dispute about pay between 2 people that is now a lawsuit. We're caught in the middle, as the IT management company. A court order exists that an employee was supposed to return their work laptop. The owner said they didn't. I had an alert where in Ninja RMM saw the laptop turn on, send an email to me. AHA, finally, time to nuke it.

I got a call on lunch: wrong laptop. UM WHAT?! First of all, they were lying. It had already been sent back. I didn't compare serial numbers to the court order because their company has 7 computers in Ninja and 2 are servers. Also, this is the one that had the ex employee's username as the "last logged in." You wouldn't check further either and you know it lol.

So I remote nuked it. Script works perfectly btw. Strongly recommended! VERY clever!
Intune/Remote-Lock.ps1 at main · HankMardukasNY/Intune · GitHub
Intune/Remote-Unlock.ps1 at main · HankMardukasNY/Intune · GitHub

We wanted to prevent access to the local copies of the Outlook emails as soon as possible! So when I saw it was still online and responding after 60 seconds of sending the script, (and I appended a shutdown command to the script), I assumed it failed and sent the backup "destroy the boot loader" script.

It was running windows updates during the shutdown. That's why it was still responding. Luckily the syntax was wrong because AI wrote the command and I didn't have time to test it, as testing it would destroy a computer. Or it's not compatible with 25h2 or something.

Anyway, employee calls in and says we locked the wrong laptop and that it's her personal laptop. HAHAHA not falling for that one, you manipulative villian! I have the receipts!

I check. It's Windows 11 Home, HP 15 series. Why TF is that in Ninja?! Oh, her work laptop broke so we put ninja on this one so she could use her personal one to access work stuff one time like 3 years ago and nobody undid it. Fantastic.

So, I disabled her personal laptop. Awesome. And she likes suing people. Well, through some Twilight Zone level circumstances that I can and would defend in court, that's what happened.

Employee was very understanding about it, especially the way I phrased what happened and how and why. Very nice lady actually. I hope she wins the lawsuit. She even said "yeah, I can see why having it enrolled in your management thing would be misleading. That was my bad." and I'm like, "UH NO, I'm the one who screwed up BADLY!" but didn't say that, cause she likes suing people.

But now they know what I look like, so I have to wear a disguise if I go to the court hearing and sit in the gallery. Darn. I wanted to see who won. This is a very engaging soap opera so far with lots of half-truths and twists and turns.

Yay, another RDP post. Anyway, one of our clients wants to use RDP for some reason to connect to their desktop from a laptop offsite. We already have Ninja Remote set up but sure, why not.

We've got computer A running 25H2 all latest updates. Same for computer B.
B is a laptop, wants to RDP into 25H2 once it's on the VPN.

We try to RDP into CompA by IP address, no connection, no response. Try hostname, nope.
In the registry, it's indeed still bound to port 3389
We allowed the user by username in RDP config.
RDP connections are turned on.
Terminal service is running
Outgoing RDP connections from computer A work just fine to other computers on their network.
10000 other checks are all as you'd expect.
Firewall rules say allow, etc etc etc.

But when I run netstat -an, there's no entry for port 3389. So nothing is listening on that port. WTF? That rules out external switch VLANs, firewalls, whatever, I guess.

Also, we completely turned off the windows firewall, same result. Zero failed login attempts seen in the Windows Security log on the target computer. It didn't see anything because it wasn't listening.

Now we're not using an RDP file, we just pull up the RDP application in windows and type in the IP address and hit connect. But still, we're not seeing that warning popup from the new update. I put in the reg fix for that anyway, no difference.

I think this is actually unrelated to the Windows update. Except all 10 of our newly imaged computers are refusing RDP connections and it works fine on every other system they own (which may be 24h2). So now they're blaming us. Someone set up the PCs before I worked here so maybe they did sabotage port 3389. I dunno.

I'm at a loss for how to fix or even diagnose this. Ran SFC and DISM and are waiting on an overnight reboot to re-test tomorrow but I guarantee there won't be a listener on 3389 tomorrow because there's no way 10 computers all randomly broke in the same way.

Does this still sound that like April 2026 update or something different and has anyone ran into this? According to my research, listening on 3389 in a fundamental part of the TS system and if it's not there, it's not repairable. So that would suck.

Just found out why our client at this MSP can't log in to their own sharepoint private site (aka onedrive). Their entire sharepoint site is blocked for phishing by the latest definitons of Fortiguard. By the way, if you ever want to check how the content on a site is classified by them:
https://www.fortiguard.com/webfilter
Anyway, I requested re-review. Anyone done this before and have a success rate % estimate and an average turnaround time?