u/Archivist_Goals

"dataindex.us is a collaborative effort dedicated to monitoring changes in federal datasets. Our team is building tools to track a broad range of changes in federal datasets to build actionable insights for policymakers, advocates, journalists, and data users."

It may be beyond the scope of some here. But data enthusiasts alike will appreciate and can perhaps monitor on their own for even the smallest, subtle changes or erasures that might happen on any of the .gov websites, and notify dataindex.us so they can add it to their tracker.

***

Per the recent article from The Guardian, https://www.theguardian.com/us-news/ng-interactive/2026/aug/18/trump-federal-data-deleted-altered

>“We’re at a very vulnerable point as a nation,” said Denice Ross, former US chief data scientist under the Biden administration and a co-founder of dataindex.us. “We’re facing a future where we are not going to have the information that we need to make decisions that will save lives and benefit American communities.”

>The tracker includes detailed information about deleted or altered datasets from more than 60 federal agencies that span the environment, labor, law enforcement and other topics. But datasets on health appear to be the most affected by the Trump administration’s crackdown, comprising about 40% of the total number of terminated or altered data collections tracked by dataindex.us.

>Multiple Centers for Disease Control and Prevention (CDC) surveys, for example, have removed questions about respondents’ gender identity. These include the National HIV Surveillance System, the primary source of case data for HIV infections, the National Violent Death Reporting System, the CDC Diabetes Prevention Recognition Program, the National Intimate Partner and Sexual Violence Survey and specific surveys evaluating Prep uptake among gay men and in communities of color.

>In addition, there are at least four health datasets that are no longer being updated at all, including the Substance Abuse and Mental Health Administration’s (Samhsa) Drug Abuse Warning Network, a surveillance system that doctors and public health researchers used to track drug-related emergency room visits in near real-time to spot emerging overdose trends.

>“These data are critical for identifying emerging epidemics and what parts of the population are most vulnerable to different types of health conditions,” said Thomas Babor, a professor emeritus at the University of Connecticut who specializes in public health and addiction.

>“We need to know what direction the trend is going, whether it’s the beginning of an end; we need to learn from prior episodes of epidemics and how they were responded to and what impact the responses had,” Babor said. “Taking down, even temporarily, some of the most important surveys that we have, especially for comparative purposes, is like we’re erasing our institutional memory for public health.”

>Other deleted datasets include the US Department of Agriculture’s twice-annual survey of farm employers that counted the number of farm workers employed and their wages that informed the minimum pay rates for H-2A visa workers and a National Center for Education Statistics (NCES) survey of the pay and working conditions of teachers and principals at public and private schools across the country."

Edit: This concerns Macrium Reflect's cloning feature only. Reflect's own image backups are different: when verification is enabled, Reflect calculates an MD5 hash for each data block before writing it to the image, then rereads the image and compares those hashes. But MD5 is the only hash it uses. Make of that what you will.

A few pointed out how it might seem counterintuitive to try and clone an OS drive and expect high-integrity validation - it uses VSS as a point-in-time snapshot - but I agree. That is silly. But even with an offline clone via Rescue Media, it still does not perform post-clone verification or reread the destination drive data and then compare it to the source. That said, I do not think MR/Paramount Software is "bad". I do think some of the terminology they use in the software is misleading.

***
For a long time, I had assumed that cloning Source Drive A to Clone Drive B would incorporate a checksum calculation during the "verification" phase that it performs. But apparently, this is not the case. I'm currently running Reflect Workstation v10.

Wanting to take fixity of data more seriously lately, it dawned on me what "verifying" actually meant when a clone job completes. So, I reached out to Macrium support, and they set the record straight: They don't verify data with any kind of fixity.

You could hash and compare after the fact. But the live source/OS VSS will continue changing, so hashing it afterward would not necessarily represent the state Macrium cloned. And I suppose one could create a per-file hash manifest. But that's not very elegant and far more work.

I've been paying for and using MR for over a decade and have been using it without issue. But this seems like a real deal-breaker. Suggestions? Any other serious alternatives?

The full exchange with Macrium support below. Happy to provide email screenshots if necessary.

>Me: At any point during cloning, does Reflect calculate a checksum, CRC, or cryptographic hash over data read from the source or written to the destination?

>Macrium: No, we don't verify the data in any way.

>Me: If so, what algorithm and granularity are used—sector, cluster, block, partition, or whole disk?

>Macrium: Not applicable.

>Me: During the stage displayed as "Verifying" or "Verifying File System," does Reflect physically reread the destination and compare it with the source or with previously calculated values?

>Macrium: No. As the name implies, it verifies the supported source file system.

>Me: If the destination is reread, does this cover every copied sector or cluster, and what happens when a mismatch occurs?

>Macrium: For RDC, both the source and destination file systems are read to determine which data has been modified. Only the modified data is then read from the source and copied to the destination for RDC. No other data is reread from the destination.

>Me: Does a successful Forensic Sector Copy certify byte-for-byte identity? Exactly which address ranges are covered: partition contents, unused filesystem space, inter-partition/unallocated sectors, partition-table structures, and any HPA/DCO areas?

>Macrium: For the Forensic option, the clone is created from the start of the partition to the end, sector by sector.

>Me: Does the verification behavior differ when cloning under full Windows/VSS versus Rescue Media?

>Macrium: No, both verify the supported source file system. In normal Windows, the VSS volume is used for supported file systems, whereas direct volume access is used in Rescue Media.

Hi there! I have a number of optical media items I have archived to Internet Archive, as part of an overall larger project. These include many DVD video disc items - think: out of print international documentaries, compact audio discs, software - and while I have the correct metadata in appropriate fields, syntax and formatting for item titles, descriptions, etc, I'd also like to include Worldcat entry links for each object.

Dublin Core / Archive.org's own guidance; RDA plus LC authority practice guides cataloging content; MARC has informed display conventions -descriptions, metadata keywords, etc.

I'm at a local medical college here in Philadelphia. And unfortunately, they do not subscribe to FirstSearch or WC Discovery.

I'm not sure to what extent WC even catalogs software - commercial, computer games, shareware - but if something doesn't show up in the general free WC search using a regular free account, does it not exist?

Is it possible to request access to one of the two catalog services without an org subscription?

Thanks!

Edit: To give people some idea, this item in particular has it right: https://archive.org/details/the-science-network-beyond-belief-enlightenment-2.0-dvd-video-2008

The description takes into account the WorldCat entry, title, authors, summary of item, format, language, publisher, pub date, physical description, OCLC #, and subjects. And, ideally, what I want to do is find every item in WC and link to its entry in the item description, just like this one does.

Edit: Turns out this was what I thought it was. The backup folder growing over time. Support confirmed the only way to resolve this is to delete the backup (Backblaze) and start over. Re-back up the data.

I’m running Backblaze Personal Backup 10.0.2.1047 on Windows. In the last few weeks, I've noticed that my Drive C space has increasingly shrunk to just a few GBs of free space.

I took a look and C:\ProgramData\Backblaze\bzdata\bzbackup\bzdatacenter contains 2,556 files named like: aside_bz_done_20260805_0.dat_safety_fix_25164.aside

It seems like, on average, something has been growing at about 7GB per day. I also looked at bzcurrentlargefile, but it appears to be empty.

Does anyone know what the safety_fix routine is doing, why these .aside snapshots are not being cleaned up and/or how I can fix this?

Or is nobody taking the fire particulate inhalation health risks seriously? Plenty of people are out, especially in CC, and only mask wearers here/there. Looks like health care workers know what's up. Beyond that, seems sparse. Yes, I am aware that blue surgical masks are not exactly robust. Yes, I'm aware that people have mask fatigue from those years.

I saw one guy wearing a full-on respirator, and other people wearing surgical masks. A few K-95s. But most are not taking this seriously. Is it just me? Have people not learned that fine particulates in the atmosphere can cause serious damage, especially to those at risk?

Edit: To the asshat downvoting comments because you somehow think masks are oppressive, your day will come. Nobody is infallible.

Edit #2: I don't normally make these sorts of posts. I use Reddit primarily for non-controversial reasons, and typically don't engage in convos that can spiral quickly into a cesspit of culture war nonsense. That said, thanks to everyone who had constructive responses and comments.

Just curious how many people in their orgs or institutions are directed to follow the BagIt 1.0 spec if you're working within DAM systems?

I'm uploading, or rather, repacking data I have previously archived to Archive.org so that it better aligns with the 1.0 standard. It's not going to be aligned in the most strict sense, as I am packing it into a non-compressed .7z, e.g., no compression, just stored. But feedback on this would be welcome! And I would love to hear what others are doing who work or manage DAMs in this respect.

Example:

<identifier>--disc-image-data.7z
  └── bag/                                                                                                                
      ├── bagit.txt                                                                                                       
      ├── bag-info.txt                                                                                                    
      ├── manifest-sha1.txt                                                                                               
      ├── manifest-sha256.txt                                                                                             
      ├── tagmanifest-sha1.txt                                                                                            
      ├── tagmanifest-sha256.txt                                                                                          
      └── data/                                                                                                           
          └── payload-root/                                                                                               
              ├── disc-label.tif                                                                                          
              ├── booklet-page-001.jpg                                                                                    
              ├── movie-title.mkv                                                                                         
              ├── disc-image.iso                                                                                          
              ├── submissionInfo.txt                                                                                      
              ├── submissionInfo.json.gz                                                                                  
              └── logs/                                                                                                   
                  ├── redumper.log 

This post is probably better served on Datahoarder. But a thought occurred after reading some horror stories of those who have lost account access - some who were Gmail account holders for more than a decade or two.

-I already archive my email locally with MailStore, and I have those files backed up in various places. The email archiving is done 1x day, every day, via an automated script. So, I rarely have to think about that. I do the same thing with Chrome bookmarks. They're synced. But I use a script that exports bookmarks every 3 days to a local folder (which itself is backed up elsewhere).

-I will also occasionally (every 6 months or so) export with Takeout in the highest fidelity settings and formats available. Anything pushed to Photos or Drive is also sitting locally on my desktop, and that data, too, is backed up. Yes - I'm aware that Takeout is not exactly perfect.

I don't necessarily have anything important from the PlayStore, aside from apps that I use, and then use my Gmail email to sign into those with. Any transaction history would likely be included in Takeout.

-But "Sign in with Google" dependencies would be annoying to lose access. And it would be a good idea if I audited just what's using that and how many, assuming recovery options for those services/sites/apps would not be sufficient enough to regain access without the email address.

Thing I am less sure about:

-Gmail settings, not just Gmail messages: Labels, Filters, Blocked addresses, Send mail as/aliases, POP/IMAP settings, etc.

-Any devices I have associated with said Google account. Obviously, any device/phone backups, call & sms history (I presume that would be in Takeout, though, I think?) I don't know what exactly I would do with the phone backup data from Takeout/how I would import that or make use of it.

I'm wondering if it would be wise to set up a separate email account on a different provider where all email from Gmail is automatically forwarded to the secondary email address. E.g., if a Gmail account is suddenly restricted or deactivated, does the user continue to receive new emails, or do they bounce back? If there's anything else I should be doing, I'd welcome the feedback. Thanks!