r/sysadmin

I am a 25 y.o mid level engineer in an older classic on prem infra team (average age around 45) and we manage a nice mix of Linux / Windows servers.
We are also in business critical so we can't just blindly copy and paste data into the LLM of our choice (like other teams in our org do), so my coworkers experience was a bit limited.
I love my job, I love being technical and I love working with my team, until recently...

After making fun of our customers for the last 2 years because they are requesting ridiculous features with the reasoning "but chatgpt/gemini/copilot said it is easy" I had a meeting with my manager about an incident that I thought was solved. He looked at me and said "yeah, well I ran it through gemini and gemini says this" and he just drops me a 1000+ word (??) answer in our chat. He didn't read it to me. He did not explain it to me. He just said "yeah that should solve it".

I looked at him like a sheep in the rain.

I read the text and just asked him if he could explain what he wanted me to follow up on, as I did not want to just forward his gemini slop (that I do not even understand).

He just looked at me like a sheep in the rain.

"Just ask gemini to explain it to you if you do not understand it?"

This man, who I have learned a lot from, has made a 180 degree turn after always explaining everything and taking the time and moved on to "just ask gemini?". The worst part is he fully expected me to just blindly copy and paste his nIcElY pReFoRmAtEd ReSpOnSe to the team dealing with the incident?

I don't know if I am just not accepting the facts, am too young to understand corporate politics and behavior or LLMs are turning people (that are smart and capable) into idiots.

Bonus highlight:
After coming back from vacation one of my coworkers and me were talking and discussion started about an upcoming project. I explained the whole architecture to him and how everything works and asked him if he can look up a flag for a CLI tool to get some benchmarks on the white board.

"Can you write me a prompt for that?"

This man just asked me, after I spent 30 minutes explaining everything to him, if I could write him a prompt? To find a flag? For a CLI tool? What happened to using google or reading documentation?
He then proceeded to show me his "research" that he did while I was gone which was just a chat with gemini? Half of the stuff was hallucinated 5 chats into the topic. The conclusions were wrong. And when we tried stuff I told him "oh this will be a waste of time, this will be 2x slower", the answer I get is "no, gemini says it will be better".
It ends up being 2.2x slower and he just looks at me like a sheep in the rain.

"bUt GeMiNi SaId It WiLl Be FaStEr"

How can I explain to these people that LLMs are very useful tools that need to be double checked and not blindly trusted? These are not dumb people, they are very knowledgable peers that taught me a lot but turned into blindly copy pasting commands, configs and spreading the information they get "with their research".

Don't get me started on their revolutionising open claw ideas...

Edit: wow that is a lot of engagement, I just wanted to rant it out - thanks for all the laughs reading the comments

Edit2: I asked gemini if it knows the idiom like a sheep in the rain and can confirm this post as well as all the comments are now in its dataset

We are getting increasingly frequent requests along the lines of “I have developed a custom application that will be a dashboard for company employees. Can you install this version of Python, an application SDK, and give an account access to our company’s financial file.”

Apparently everyone thinks they can code. Needless to say, I have not seen one of these ideas come to fruition in the form of a production-ready application.

I am curious how others are handling these requests. I have no interest in facilitating this behavior if it can be avoided.

We're a growing law firm in Australia (currently around 10 staff, likely 15+ within the next couple of years) and I'm looking to standardise our laptops.

I'm currently leaning towards Lenovo ThinkPads (likely T14s or similar) because they seem to have a strong reputation for reliability.

My biggest concern isn't actually the hardware itself it's support. If a solicitor's laptop dies before a court appearance, mediation, or client meeting, downtime is incredibly expensive.

For those managing business fleets in Australia:

- How has Lenovo Premier Support been in practice?
- If a ThinkPad fails, how quickly are repairs actually completed?
- Has anyone had experience with replacements being provided?
- Would you choose Lenovo again, or would you go Dell Latitude + ProSupport instead?

Also, what's the best way to purchase and manage these?

- Do you buy direct from Lenovo or through a reseller?
- Should I be engaging an IT provider to source and manage the fleet?
- Is there anything you'd do differently if you were setting up a fleet of 10–20 laptops from scratch today?

I'm less interested in benchmarks and more interested in experiences when things go wrong.

Located in Australia if that makes a difference.

TIA

Just a quick poke to see if anyone else has had an influx of crashing/black screen freezes.

We're a HP house seeing most of the issues on our elitebook 8 G1is and elite book 840 G11 models

Users report that it happens randomly, with some doing a BSOD.

Over the weekend we’ve been forced onto Microsoft’s latest link-sharing model.
Now instead of sharing a link and have the external simply access their file with a single-use code, we now have to have them create a guest entity in our Entra, set up our MFA method and above all else we have to manage these externals - and we have A LOT of them.

I get why this change has been made but IT teams globally now have to manage externals and support them.

Now we have an onslaught of tickets coming in about how externals are finding it impossible to access files shared with them.

So far I’ve written up a guide specifically for external use that our users are to share when they send a link to a new external.

What have you done to ease the burden? Anything to make this whole change less gruelling?

Cheers

Question to you sysadmins out there:

Would you use Azure Update for everything, just as it is, leave it update systems automatically? Would you rely on it?

Why I am asking:

I recently joined a company as a senior admin, and honestly, have my doubts. In my previous company, we exactly declined to use Azure Update due to it's uncontrolled behavior. You basically tell it go... and it will reboot at time it chooses... or maybe even? It's kinda like: "trust Microsoft". Which I don't, honestly.

The company I am in is "quite happy with it". Until I heard that the Linux systems were patched, but not rebooted... so CopyFail was still unpatched everywhere (due to patched kernel not being active). Gave me chills.

Personally, I am on the level of using what's best on both systems, be that Windows or Linux. For Linux I'd take nothing else but SemaphoreUI + Ansible. So much native control. For Windows, meh... either Ansible (works), or any other tool out there that does Windows Patching. The point being - I am in the control, what happens and when!

So, what is your stance towards Azure Update? Would you use it? Would you not? Would gladly hear your reasons for both why yes or why not!

Thanks

EDIT: I want to clarify that I looked last at Azure Update Manager last at least half a year ago. I have no *current* experience, so I might be out of date. This is merely about your opinion and experience.

At $300k/month Cloud spend, our bill keeps 
growing faster than our traffic.

Cost Explorer shows the numbers but nobody 
actually checks it weekly.

Trusted Advisor gives 40+ recommendations 
with no priority order.

Anomaly detection emails get archived.

What actually works for your team?

Curious about:
- How often someone reviews the bill
- Whether you automate any cleanup
- If you bought a tool, which one and is it used
- War stories from cost incidents

Trying to learn from teams that figured this out.

I am trying to properly deploy defender for endpoint on a macOS - but the instructions I see seem to be very manual "easy but manual" and none of them reference the area within intune - Endpoint Security > Manage > Antivirus - and creating MacOS policies there - also, if Endpoint Security is the way to go, which one do I deploy? - If I pick MacOS, I get 3 templates - one is exclusions "I know what that does", the other two are MacOS Endpoint Security AV and Microsoft Defender Antivirus. Thoughts? I already posted this to the /intune sub and have had zero response, which makes me think everyone is doing something different.

Good morning. I'm looking for a solution to put switches in HA. That is to say, connect a server for example on each of the switches and if the first switch falls it continues seamlessly on the second. Do you have any ideas for a solution ?

I recently got a new laptop with a snapdragon processor running W11 Pro and cannot for the life of me install the RSAT tools, specifically AD U and C. I've tried adding thru the optional features, adding through Powershell, but nothing ever works even with the assistance of Gemini. Has anyone found a way? Here are my OS details:

Edition Windows 11 Pro

Version 25H2

Installed on ‎5/‎20/‎2026

OS build 26200.8457

Experience Windows Feature Experience Pack 1000.26100.304.0

Processor Snapdragon(R) X Plus - X1P42100 - Qualcomm(R) Oryon(TM) CPU (3.24 GHz)

System type 64-bit operating system, ARM-based processor

I have configured (on VirtualBox):

1. A Windows Server 2019 as Domain Controller
2. A Windows 10 VM as a Workstation

From my DC, I have configured a GPO (One single policy):
1.

Computer Configuration > Policies > Windows Settings > Security Settings > Advanced Audit Policy Configuration > Audit Policies > Detailed Tracking-> Audit Process Creation -> Checked both Success and Failure

2.

Computer Configuration > Policies > Administrative Templates > System > Audit Process Creation -> Include command line in process creation events. -> Enabled

I have further configured (To ensure legacy audit policy does not override advanced audit policies):

 Computer Configuration → Policies → Windows Settings → Security Settings
→ Local Policies → Security Options --> Audit: Force audit policy subcategory settings (Windows Vista or later) to override audit policy category settings -> Define this policy setting --> Enabled

Finally, to ensure that, Fast Boot is not messing foreground synchronous GPO processing of my workstation, I configured Default Domain Policy:

Computer Configuration > Policies > Administrative Templates > System > Logon -> Always wait for the network at computer startup and logon -> Enabled

But when I run this from workstation:
gpupdate /force

and then:
auditpol /get /subcategory:"Process Creation"

I get this:
No Audit

However,
gpresult /r /scope computer
shows that the custom policy is downloaded

I have already combed through google and AI chatbots for this, but found no solution. I'd appreciate any guidance regarding this.

Occasionally read stories in job-related subs about employees transferring to other departments within their employer.

They usually don't say what they did and where they're going, but presumably the employee had skills that the receiving department manager felt could easily transfer to that department.

Off the top of my head, sales to marketing and vice versa could be a natural transition. Design to sales perhaps.

I've been in IT operations for over 20 years, from office, to government to (currently) industrial manufacturing. I can't really think of any departments - at least within my company - that I'd be suited for. And at this point, I'm not starting over.

Coming into IT perhaps, but leaving; I guess myself I'm feeling pigeon-holed. Don't really care as I like what I do, but seeing doom and gloom in the job markets has made me wonder just how marketable I can be if the IT sector totally fell to shit.

So we are in a M365 environment. We have DKIM records for our domains and they work fine. However, we also have a third-party vendor that has a service that sends emails on our behalf. Some of these emails have been bouncing back because Microsoft said they lack the required DKIM record. The vendor claims that it would be something to configure on our end, not theirs.

I'll admit my understanding on DKIM is limited, but if they are sending on behalf of our domain/emails, wouldn't they need a record on their end showing that they're authorized to do so?

Hi, all. I'm working on automating a legacy Windows desktop application (built on an old Gupta SQLWindows framework) on a headless Ubuntu server. I finally got it working after repeatedly fixing dependencies, which has me questioning if this is the right approach.

I want to know if headless GUI automation via Wine is a standard industry pattern for this scenario, or if there is a better approach I overlooked?

Some more context:

Because of our infrastructure bias, the pipeline must run on Linux servers. To do this headlessly, I built out:

• Ubuntu Server running Xvfb to handle the graphical rendering layer
• 32-bit Wine prefix running an isolated, embedded Windows Python 3.10 instance
• Dropped down to raw Win32 API hooks

Initially, I tried using modern Python libraries like pywinauto and pandas for handling extracted data manipulation but this created more errors.

• I ran into UCRT crashes due to missing math hooks inside Wine. I had to take out pandas/numpy entirely and rewrite my script using the native Python csv module
• since Xvfb is an invisible memory display layer, traditional background macrotriggers threw COM errors because they can't grab physical system foreground focus. I had to switch to scan-code injection to bypass window focus constraints
• headless winetricks installers panic without a visual display engine, meaning I had to manually use cabextract to rip old Visual C++ components (mfc42.dll) out of Microsoft cabinet setup caches and register them by hand.

For those who've had to host legacy, closed-source Windows desktop apps on Linux infra, is wine+xvfb+win32 hardware sim the standard procedure?

Are there any better approaches? Assuming a rewrite of the source app and commercial RPA's (not enough use cases) are off the table what else can I do? ie: docker on windows server nodes?
Is there an open-source toolchain better suited for headless Windows-on-Linux UI interaction than raw Win32 API calls via Python

I am a business analyst working for a nonprofit of ~2000 employees, which increases to closer to 3000 during summertime as we have a lot of seasonal positions. I was hired specifically to support our HR department because I have several years experience as an analyst in leave and accommodations. I knew at hire that there were plans to potentially roll out a new HRIS and I took the job. I was upfront that I do not have HRIS configuration experience but that I want to learn, and I think this will be an excellent experience.

I knew going in that I would be heavily involved in the implementation, and sure enough, a contract was signed between my offer acceptance and the start date. I had about a month's lull before the project got into full swing to get acquainted with our systems, teams, and processes. I also learned very quickly that things are a mess, everyone is stressed, and capacity is thin, which I was already primed for, knowing this is nonprofit sector work. However, things have gotten crazier, and I feel like I need a sanity check from someone not in the middle of this.

We are now about 5 months into our implementation with a go-live date in the fall. In March, our HRIS director quit. No replacement has been hired yet, and it took about a month before we were clearly told that leadership has no intention to hire a replacement. At the start of May, our HRVP quit. Both were major decision makers on the project.

Since this HRVP left, it's now myself and this HRIS admin who are being looked to for decisions when we hit sticking points. These include things like determining CRUD authorizations and role access as well as providing signoffs on configuration testing. On paper, my VP (VP of IS) is signing off, but she's not present and basically has no idea what's going on. Periodically, she's stepped in to demand additional approval, which largely means we then have to have half a dozen meetings just to bring her up to speed on the current context, and it wastes a ton of time.

Until a few weeks ago, we didn't even have an internal project manager; we went through two contracted PMs, the first of whom was an asshole, and the second of whom was juggling multiple clients and had limited capacity to provide support. He's been a massive help and has taken a lot of administrative load off my shoulders, as I was also being looked to as the primary POC for scheduling meetings and coordinating SMEs for the different modules across HR.

This brings me to our current state. Our "HRIS" department is me, that one HR admin who actually just went on leave because she's understandably overwhelmed by all this, and a temp who is being pulled more into the project simply due to lack of capacity from those already involved in it. I communicated my own frustration of how thin we're all being stretched to the VP today. I told her that we need leadership involved in this project that has the authority to direct some of this decision making at a high level, even if it isn't a director level role like the person who left in March. She basically said to me that it was always the plan for a BA role to do the job that this HRIS director was doing, never mind the fact that I'm paid a good 40k less and have been with the company for just barely over six months.

She essentially said that she wants me to be the HRIS admin AND a business analyst at the same time, while ALSO being the primary contact on this implementation project. Which, again, is my FIRST major system implementation. My first month on the job, I had started evaluating areas that needed major support (we have no system for LOA, a ton of manual processes to pay benefits including some that still use paper forms, and about a bajillion excel trackers that are all reworked every year or two when staff turnover destroys all institutional knowledge). I joked that I would never run out of things to do, no matter how many processes I automated. Joke's on me, because all those projects have ground to a halt, and are unlikely to ever resume if the entirety of HRIS actually gets dumped onto my plate.

I knew nonprofit would be a bit crazy. Is this anything like normal nonprofit levels of crazy, or am I being thoroughly taken advantage of?

Minor edits made for clarity.

Sorry. Feel free to delete mods. Just needed to yell this to the void without getting fired.

what is your workplace guideline on replacing laptops and getting new ones for employees?

GitHub Official X Post

"We are investigating unauthorized access to GitHub’s internal repositories. While we currently have no evidence of impact to customer information stored outside of GitHub’s internal repositories (such as our customers’ enterprises, organizations, and repositories), we are closely monitoring our infrastructure for follow-on activity."

Dark Web Informer says "GitHub source code allegedly offered for sale: Internal orgs and private repositories claimed

A threat actor using the alias TeamPCP claims to be selling GitHub source code and internal organization data.

The actor claims the dataset includes around 4,000 private repositories and says samples can be provided to interested buyers to verify authenticity.

━━━━━━━━━━━━━━━━━━━━
Target: GitHub
Country: United States
Sector: Technology / Software Development / Source Code
Incident Type: Alleged Source Code Sale
Claimed Exposure: Around 4,000 private repositories
Actor: TeamPCP
Price: Offers over $50,000
━━━━━━━━━━━━━━━━━━━━"

Edit: adding xcancel link, thanks jykke!

Update from GitHub:

1/ We are sharing additional details regarding our investigation into
unauthorized access to GitHub's internal repositories.
Yesterday we detected and contained a compromise of an employee device
involving a poisoned VS Code extension. We removed the malicious extension
version, isolated the endpoint, and began incident response immediately.

2/ Our current assessment is that the activity involved extiltration of GitHub-
internal repositories only. The attacker's current claims of ~3,800 repositories
are directionally consistent with our investigation so far.

3/ We moved quickly to reduce risk. Critical secrets were rotated yesterday and
overnight with the highest-impact credentials prioritized first.

4/ We continue to analyze logs, validate secret rotation, and monitor for any
follow-on activity. We will take additional action as the investigation warrants.

5/ We will publish a fuller report once the investigation is complete.