u/bayinfosys_ed

My hypothesis is that non-human requests can be identified by non-human review with a good collection strategy. This is how five days worth of data was interpreted by claude:

1,849 records, 181 distinct user agents

• User-initiated fetches.
  • ChatGPT-User fetched two pages (5 October 19:17 and 6 October 17:30), with no referer. One minute after the first, OAI-SearchBot began a 30-path crawl that lasted an hour.
  • Google-NotebookLM made one fetch on 4 October.
• ClaudeBot. It made 59 requests, all to robots.txt and sitemap.xml (2 distinct paths), and never fetched a page.
• Meta. 190 requests are almost all with a referer.
• GPTBot. Seven requests, first seen 4 October 08:28.
• Bursts.
  • 277 requests in two minutes on 5 October at 00:42, from Chrome 131 on Linux, across 272 distinct paths, with no referer. That pattern fits a path scanner. The status column would confirm it.
  • 34 requests from Chrome 128 with OPR/114 in two minutes at 23:48 that night, across 32 paths.
  • 38 AhrefsSiteAudit requests in one minute (an audit run, separate from AhrefsBot).
• Scanners and profilers. Palo Alto (25), Censys (7), MerchantSecurityScanner (2), BuiltWith (2) and gitscan (2).
• Google tools.Google-InspectionTool made 9 requests and Googlebot appeared between 19:49 and 19:53 today. If you ran a live test in Search Console around 20:50 BST, those are yours (NB: I did do this).
• Stale browser strings. Chrome 48, 57, 76, 80, 85, 89, 99 and 113 appear.

This is a new feature of https://clientlog.bayis.co.uk which already captures client-side requests from privacy respecting beacon script.

Hi all, how do you deal with fine-tuned models having different EOS/special tokens compared with the base model?

When I download fine-tuned models from HuggingFace, I often find the EOS token (and other special tokens) have been changed from the base model's defaults. This makes it fiddly to run these models generically, since I can't just assume the standard tokens will work.

How do you handle this? Do you check the tokenizer config for every model before running it, or is there a more automated way to detect/handle these mismatches?

Hi all, how do you deal with fine-tuned models having different EOS/special tokens?

When I download fine-tuned models from HuggingFace, I often find the EOS token (and other special tokens) have been changed from the base model's defaults. This makes it fiddly to run these models generically, since I can't just assume the standard tokens will work.

How do you handle this? Do you check the tokenizer config for every model before running it, or is there a more automated way to detect/handle these mismatches?

A scaled and accurate rendering of the solar system in 2D: we have the various systems (earth, jupiter, saturn, etc) with lagrange points, apoapsis, periapsis of orbits.

Hello all,

I have a freemium offering, running in the background making no money (it is very useful as a portfolio piece, so i would like to keep it alive). After signup the user gets two free goes, then has to pay for more.

Recently, it's come under attack from someone looking to exploit the freemium process. The attack is simply fake signup by bots using old/inaccessible email accounts which bounce the verification email. The architecture of the setup means they won't get any value, but it could still cause problems by running up charges.

Curious what you guys would do from a product perspective. For now I have removed the freemium offering, but it doesn't feel good. I enjoy building publicly visible services, but this kind of thing introduces some challenges. wwyd?

(NB: The system is hosted on AWS, with a HTTP API rather than REST API - so no WAF protection.)

I've built various systems, services, servers, etc over the years, but never really settled on how to monitor power draw. Is there a best practice? Something you would recommend?

I would specifically like something between the PSU and the wall socket which captured current/voltage consistently and sent it to an endpoint in a common format - without enterprise prices!

Hi all, can anyone shed light on why backlink sites exist?

They get marked as spam, they have no useful content, I don't get it. Are they targeting crawlers somehow? paid by bandwidth?

I just don't understand the incentive!