u/Ilia0001

r/technicalanalysis r/ClaudeCode r/claudeskills r/laravel r/pinescript r/PHP

Nine C extensions, one release cycle: more time spent on perf regressions from my own safety checks than on features

All nine shipped this week, and almost none of it is new features. I posted a roundup here earlier this month arguing that most of the work in these extensions is hardening rather than features. This is the follow-on, and the bill for that hardening came due in a way I did not expect.

php_excel makes every save atomic now: stage the workbook to a temp file, rename it into place, so an interrupted write cannot destroy the file you already had. The straightforward way to build that staged file is to ask LibXL for the finished archive as one buffer and write the buffer out. That costs 67.7 MB of peak RSS on a 3.3 MB workbook, and the PHP-side copy counts against memory_limit. The streaming writer it replaced costs 0.7 MB. Roughly 20x the workbook size in RAM, spent by a change whose entire purpose was safety. That one never shipped, it was caught and fixed inside the same release window, but it sent me looking.

I found the same thing in three more places.

pdo_duckdb re-latched its open_basedir sandbox once per row and compared the recorded basedir by hash, so a per-row cost scaled with the length of your open_basedir string. It now re-latches once per fetched chunk and compares by string. A 400k-row scan went from 133 ns/row back to 51, and bulk Appender::appendRow() from 196 to 96.

fastjson ran an exact-size preflight for large strings starting at 1 MiB. Below 8 MiB the second pass cost more than the reservation it saved: 75% slower on x86_64, 160% on aarch64 for UTF-8 text. The optimization was real and the threshold was wrong.

phonetic had an optimization routing 1 to 3 element comparisons through memcmp(). That cost 5 to 6% of encode time against comparing code points inline, so it got reverted.

Some of the cost stays paid on purpose. fast_uuid's ramsey/uuid compat wrappers now validate their core on construction, one getVersion() call and a class-name compare, and construction came out about a third slower for it (fromBytes() 1.81M to 1.21M ops/s). A wrapper that does not match its core is a bug that surfaces somewhere much worse, so I kept it.

All nine are open source (mixed PHP-3.01 / BSD / MIT), free, installable via PIE, and I am the author. Full write-up with all nine changelogs: https://ilia.ws/blog/the-cost-of-failing-closed-what-shipped-across-nine-php-extensions

Do you profile after a hardening pass, or only after a feature? I have started treating "we added a check" as a perf-regression trigger, and I am not sure whether that is normal practice or paranoia.

reddit.com

u/Ilia0001 — 24 days ago

▲ 4 r/claudeskills

I built two tools for Claude Code because the agent was failing in two different ways

Two things kept breaking in my sessions, and they needed different fixes. The agent would call functions that don't exist, and edit one file while breaking three callers it never opened. Separately, it would skip planning, write tests after the fact, and declare done without checking anything ran.

The first is a knowledge problem, the second is a process problem. CodeSage covers the first: a Rust binary that builds a structural graph plus a semantic index of the repo and serves it over MCP, so find-references returns the real call graph instead of a text match. Whetstone covers the second: skills that act as gates the agent can't skip. Neither fixes the other's failure.

Running them together is what exposed the flaw in each. Reviewing three of my own extensions took 3 or 4 passes each, and one finding was simply wrong: the model "fixed" it, then caught it as bogus at its own validation step. CodeSage 0.15 now puts new findings through an adversarial verifier by default, and 0.17 drops any finding whose quoted evidence doesn't match the file it cites. Whetstone had the mirror problem: eight skills were firing on a word anywhere in the prompt rather than the intent, so merely naming a worktree path pulled in the worktree skill. All eight now require the verb near the noun.

Disclosure since both are mine: I'm the author, both are free and open source under MIT, and there is nothing paid or monetized.

https://github.com/iliaal/codesage

https://github.com/iliaal/whetstone

u/Ilia0001 — 1 month ago

▲ 30 r/PHP

Seven of my native PHP extensions shipped last month, and most of the diff wasn't the new features, it was rejecting bad input instead of storing it

I maintain seven native PHP extensions: php_excel, mdparser, php_clickhouse, fastchart, fastjson, phpser, and fast_uuid. Each shipped a release or two in the last month. I did a roundup here in June, so this is what changed since, plus one observation about where the time actually went.

The features are what you notice in a changelog. But when I counted the lines across these seven, most of them were the other kind of work: rejecting a bad value instead of storing it, failing a write cleanly instead of half-committing it, throwing on the untrusted path instead of trusting it. That is the unglamorous majority of maintaining a C extension, and it never reads well in release notes. A few concrete ones, next to the features:

php_clickhouse got native JSON and Bool column support and IPv4/IPv6 writes, plus a decoder rewrite: dropping a dynamic_pointer_cast that a profiler blamed for ~18% of decode instructions took wide-integer SELECTs ~25% faster and numeric-heavy inserts 30-44% faster. The fail-closed half: inserting PHP null into a non-Nullable JSON column now throws instead of silently storing an empty {}.

fastchart went from 26 chart types to 38 in one release (dendrograms, chord diagrams, network graphs, violin plots, Venn diagrams, word clouds), and a perf pass made a 4096-point scatter render ~6x faster and PNG output ~40% faster.

phpser, my binary serializer for cache workloads, added a columnar wire format for rowsets: it writes the column schema once, then each column as a typed run, which drops the per-row key overhead for the array-of-uniform-rows shape cache and queue payloads usually are. Rowset decode came out ~24% faster. It also closed two issues on the untrusted decode path.

fastjson can now edit a JSON document in place by RFC 6901 pointer (fastjson_pointer_set), and it ships prebuilt binaries now, so PIE installs a .so instead of source-building on Linux and macOS.

php_excel added bulk reads (readRange pulls a rectangular block of cells in one call), and the fail-closed pass: save() to a file:// path is now atomic, so a short or interrupted write no longer destroys your existing file before the new one is finished.

fast_uuid added binary and batch generators (uuid_v7_bin_batch returns raw 16-byte monotonic v7s for a BINARY(16) column), plus a set of ramsey/uuid compatibility fixes, including a COMB codec bug that made ramsey-written COMBs decode to a different UUID.

mdparser was mostly hardening this round, since the md4c engine swap already landed: a stack-buffer overflow in the CommonMark XML serializer, and an alt-text attribute escape.

All open source and installable via PIE; each extension name above links to its repo and full changelog. I wrote the whole thing up, with the fail-closed argument in more depth, here: https://ilia.ws/blog/failing-closed-what-shipped-across-seven-php-extensions

Curious whether anyone else deliberately spends more of a release on fail-closed behavior than on features, or whether that reads as over-engineering to you.

u/Ilia0001 — 1 month ago

▲ 35 r/PHP

I set out to argue against deprecating PHP's metaphone(). Then I read the RFC.

A while back an RFC landed on internals to deprecate metaphone(). My first reaction was the reflex of an old release master: leave the legacy string functions alone, people depend on them, deprecation churn is its own tax. I was ready to argue against it.

Then I read the reasoning and looked at what phonetic name matching is supposed to do in 2026, and I changed my mind. metaphone() is the original 1990 algorithm: English-only, single-key, tuned for one accent of one language. It was superseded by Double Metaphone in 2000, which emits a primary and an alternate key. Core shipped the first version and never moved. soundex() is older still, a 1918 patent. For the one job these functions exist to do, collapsing names that sound alike but are spelled differently, they are the weakest tools in the drawer. The deprecation is defensible.

Where I part ways with the RFC is the replacement. It points at userland Composer libraries. But phonetic encoding is a hot inner-loop operation, run over every name in a dataset to build a match index. A pure-PHP implementation pays interpreter overhead on every character. The honest replacement for a native C string function is another native C string function. That performance gap is the whole reason this code lived in core to begin with.

So I built the thing the RFC should have recommended: iliaal/phonetic, a native extension with the five encoders core never had, plus a comparison helper per algorithm.

double_metaphone_match("Catherine", "Kathryn");   // 2  (primary keys agree)
dm_soundex_match("Moskowitz", "Moskovitz");        // true  (one surname, two transliterations)
bmpm("Garcia", BMPM_SEPHARDIC, BMPM_EXACT);        // "garsia|gartSa"

Double Metaphone is the fast general default. Beider-Morse (BMPM) is language-aware and matches across transliterations and scripts. Daitch-Mokotoff is the genealogy standard for Eastern-European and Ashkenazi surnames. NYSIIS and Match Rating are cheap English second keys. The helpers matter because each encoder answers "do these sound alike" differently (two keys, a code set, or a threshold), which is exactly where userland code quietly gets it wrong.

BMPM is slow, roughly 60x a Double Metaphone call, so you pick it for recall, not throughput. These are heuristic, culture-bound, Latin-script encoders, not a universal global-name solver.

One aside r/PHP might appreciate: the BMPM rule data is a licensing trap. The canonical PHP reference and the abydos Python port are both GPL-3.0 because of that data. I vendored the identical tables from Apache Commons Codec under Apache-2.0 to keep the extension BSD/Apache-clean.

pie install iliaal/phonetic

Repo: https://github.com/iliaal/phonetic

Happy to answer questions, especially from anyone doing record linkage or dedup across messy person data.

reddit.com

u/Ilia0001 — 2 months ago

▲ 40 r/PHP+1 crossposts

I forked a dead PHP name parser because it couldn't tell a credential from a surname

I use theiconic/name-parser at work to split full-name strings into salutation, first name, last name, suffix, and so on. It does the boring parts well, but it has a bug that bit me on a list of clinicians: parse "Jane Doe DDS" and the last name comes back "Dds", with "Doe" shoved into the middle name. The dental credential became the surname. Almost every row with a trailing credential and no comma did some version of this. Upstream went quiet around 2020, so it never got fixed. I forked it: iliaal/nameparser.

The root cause is that upstream runs every token through strtolower() before matching it against its credential dictionary. That throws away the one signal that separates a credential from a name. People write credentials in caps and names in title case. "Smith, Ma" is a person named Ma; "Smith, MA" is a master's degree with no recorded first name. Lowercasing deletes that distinction before anything looks at it. The fork reads an ambiguous token ("Do", "Vi", "MA", roman numerals) as a credential only when it is all-caps; title case keeps it as a name. So "Jane Doe DDS" keeps "Doe" and reads "DDS" as the suffix.

It also handles international surname particles now: "van den Heuvel", "de los Santos", "vom Bruch", "le Pen", "dos Santos", "dela Cruz", and "lo Russo" keep the full surname instead of orphaning the particle into the middle name. The comma form works too ("van der Berg, Johan" gives last name "van der Berg"), and there is an opt-in setSurnameFirst(true) for comma-less CJK order ("Mao Zedong" to last "Mao").

For batch imports there is an advisory getConfidence() that flags rows where casing couldn't decide, so you can route those to manual review instead of trusting every split. It is opt-in and does not change what parse() returns.

The honest limitation: casing is the signal, so uniform-case input (all-caps legacy data, or all-lowercase) carries none. The README says so plainly. It is a heuristic, not a universal global-name solver.

It is a maintained fork, not original work: The Iconic's parser (quiet since ~2020), Zachary Miller's PHP 8.3+ modernization, and my casing, credential, and international layer on top. PHP 8.3 through 8.5, PHPStan level 9, MIT.

composer require iliaal/nameparser

https://github.com/iliaal/nameparser

Happy to answer questions, especially from anyone parsing professional or registry name data.

u/Ilia0001 — 9 days ago

▲ 55 r/PHP

pdo_duckdb: a PDO driver for DuckDB, and my first PDO driver in 15+ years

DuckDB is the in-process analytical database, roughly SQLite for OLAP work: columnar, vectorized, no server, one file. PHP has shipped PDO_SQLite in core for twenty years but had nothing equivalent for DuckDB. If you wanted DuckDB from PHP, your options were FFI against its C API or shelling out to the CLI, neither of which looks anything like the rest of your data layer.

So I wrote pdo_duckdb, a native PDO driver. You connect with a DSN and use the same PDO API you already use for SQLite, MySQL, and Postgres: prepared statements, positional and named params, transactions, foreach over results.

$db = new PDO('duckdb:/path/to/analytics.duckdb');
$stmt = $db-&gt;prepare('SELECT region, SUM(amount) AS total FROM sales WHERE year = ? GROUP BY region');
$stmt-&gt;execute([2026]);
foreach ($stmt as $row) { /* ... */ }

A few things beyond the standard surface:

a native Appender for bulk loads, much faster than row-by-row INSERT, and it accepts PHP arrays for LIST/STRUCT/MAP columns
result streaming for large scans via PDO::DUCKDB_ATTR_UNBUFFERED, so a huge SELECT isn't bounded by memory
DuckDB's own extensions (httpfs, json, and so on) load through plain SQL, no special API
when open_basedir is set, the driver disables DuckDB's SQL-level file access (read_csv, COPY, ATTACH) so the sandbox holds at the SQL layer, not just for the database file path

It's honest-early. The query path, appender, transactions, and type decoding all work and are tested, but it's young, and I haven't published benchmarks because the speed here is DuckDB's and the driver's job is to stay out of its way. lastInsertId isn't supported, since DuckDB has no rowid.

This is also the first PDO driver I've written since I was one of the original authors of PDO back around 2005. The driver model itself has barely changed; the world around it has. The thing that actually cost me a release was static-linking DuckDB's C++ runtime into a C extension so the prebuilt Linux binary loads on a clean host. PIE ships prebuilt binaries now, so install is just:

pie install iliaal/pdo_duckdb

Free, BSD-3 licensed. Repo: https://github.com/iliaal/pdo_duckdb

Longer write-up with the details: https://ilia.ws/blog/pdo-duckdb-a-pdo-driver-for-duckdb

Happy to answer questions, and I'd welcome feedback from anyone running DuckDB for analytics off a PHP stack.

u/Ilia0001 — 2 months ago

▲ 33 r/PHP

What shipped across my native PHP extensions since I last posted here, and why five of them lowered their minimum PHP version

I maintain a set of native PHP extensions: php_excel, mdparser, php_clickhouse, fastchart, fastjson, phpser. Each has had a release or three since I last posted it here, and rather than drop six separate release threads on you, here is one roundup of what actually changed.

The thing that cuts across most of them: I lowered the minimum PHP version. php_excel, fastchart and fastjson now build on 8.1; phpser and mdparser on 8.2. All of them had required 8.3. Most libraries raise their floor over time rather than lower it, but for a native extension the minimum is a packaging decision, not a language one. None of these needed an 8.3-only engine API. So if you are pinned to 8.1 or 8.2 by your distro or your employer, you can run them now.

What else shipped, briefly:

mdparser had the biggest internal change. I swapped the parsing backend from cmark-gfm to md4c, a single-file streaming parser compiled into the extension, which roughly doubled throughput: the old backend benchmarked at ~5-9x the fastest pure-PHP parsers, md4c is ~10-20x. It also brought CommonMark 0.31 conformance (652/652) and a set of opt-in dialect extensions (LaTeX math, wiki links, ==highlight==, super/subscript, GitHub-style admonitions). The public API did not change, so it is a drop-in upgrade.

php_excel added libxl 5.2.0 support, so you can now read the data validations stored in an xlsx file, and a new AS_TEXT write mode that writes a value verbatim, so untrusted input starting with '=' cannot turn into a live formula when someone opens the file. Plus a sweep of bounds checks on the libxl integer arguments.

php_clickhouse got insertFromStream(), which stream-parses a TSV or CSV file and INSERTs it in C++ batches without buffering the whole file in PHP memory. The release after was a hardening round: fixed a heap use-after-free reading Map columns, a clone-corrupts-the-heap bug, and setDatabase() now survives a reconnect instead of silently reverting to the constructor database.

fastchart changed the most by version number, 0.2 to 1.3, so it is past 1.0 now. The recent highlights are vector PDF output (renderToFile('out.pdf') renders every chart type as real vector geometry, no rasterization) and structured image-map data for click regions.

fastjson grew document-surgery functions: RFC 6901 JSON Pointer reads that pull one value out of a large document without decoding the whole thing, RFC 7386 merge-patch, and a relaxed decode mode that accepts JSONC (comments, trailing commas). All backed by yyjson.

phpser, my binary serializer aimed at cache workloads, got a faster decoder: it installs declared object properties straight into their slots instead of building a properties hashtable per object, about 22-25% faster on same-class DTO batches. It also closed a correctness gap where a crafted numeric-string array key could slip past isset() / array_key_exists().

All of these are native C extensions, BSD-licensed, installable via PIE, and live at github.com/iliaal (each repo has its full changelog). I wrote up the minimum-version decision and most of these in more depth here: https://ilia.ws/blog/lowering-the-php-floor-what-shipped-across-five-extensions

Happy to answer questions, especially on the mdparser engine swap or the php_clickhouse streaming loader, which are the two I expect people will have opinions on.

reddit.com

u/Ilia0001 — 2 months ago

▲ 66 r/PHP

fast_uuid: RFC 9562 UUIDs for PHP in pure C, 11-57x faster than ramsey/uuid

UUID generation sits on a lot of hot paths. Every ORM insert with a UUID primary key, every cache key, every event or trace ID. ramsey/uuid is the default in most PHP codebases, and it's correct and feature-complete, but it calls random_bytes() once per UUID. That syscall dominates v4 generation. At a few thousand inserts a second, the per-call cost stops being noise.

I wrote fast_uuid, a C extension (pure C, no C++/libstdc++) that generates every RFC 9562 / RFC 4122 version: 1, 2 (DCE Security), 3, 4, 5, 6, 7, 8, plus nil and max. Two things do most of the work:

Batched CSPRNG. Instead of one getrandom() per UUID, it pulls 8 KB into a per-thread buffer and amortizes one syscall across ~500 v4s. That's where most of the v4 speedup comes from.
SIMD hex formatter. 16 bytes to 32 hex chars in a handful of vector ops, runtime-dispatched: SSSE3 pshufb on x86-64, NEON vqtbl1q on ARM64, scalar fallback elsewhere. No build flags.

The object API mirrors ramsey/uuid under the FastUuid namespace. There's also a procedural zero-allocation path (uuid_v4(), uuid_v7()) that returns a zend_string with no object allocation for the hottest call sites.

Throughput vs ramsey/uuid 4.9.2 and PECL uuid 1.3.0, PHP 8.4.22 NTS non-debug, best of 40 runs, in million ops/sec (higher is better):

Operation	fast_uuid (obj)	fast_uuid (proc)	ramsey/uuid	PECL uuid
v4 gen to string	12.6	19.5	1.10	0.47
v1 gen to string	12.3	16.5	0.29	8.22
v7 gen to string	12.1	19.8	0.66	n/a
parse to 16 bytes	10.4	16.2	3.18	5.28

One honest caveat on the numbers: the fast_uuid ops are around 50 ns each, so scheduler noise dominates a single run. Read those columns as order-of-magnitude (roughly plus or minus 10 percent), not three significant figures. ramsey/uuid (~900 ns) and PECL (~2 us) reproduce to within ~3 percent. Speedups land at v4 11-18x, v1 42-57x, v7 18-30x, parse 3-5x.

A couple more caveats worth stating up front. The ramsey-compatible layer (FastUuid\Compat) is not on Packagist yet, so today you install it as a Composer path repository. Migration is mostly a use-swap, but it's not a composer require away. uuid_v4_fast() uses a non-cryptographic xoshiro256** PRNG, so it's for non-security IDs only. And supplying your own RandomGenerator or TimeGenerator routes generation off the C fast path by design, the same way ramsey behaves.

If you're moving to UUIDv7 for time-ordered keys: v7 here carries sub-millisecond precision (RFC 9562 6.2 Method 3), so same-millisecond v7s still sort in order, and an integer-millisecond API skips DateTime construction entirely.

Full write-up with the methodology and the ARM64/NEON numbers: https://ilia.ws/blog/i-generate-too-many-uuids-so-i-wrote-a-faster-one

Install: pie install iliaal/fast_uuid (prebuilt binaries for Windows x86/x64, Linux glibc x86_64/arm64, macOS arm64). PHP 8.1 through 8.6, NTS or ZTS, BSD-3-Clause.

Repo: https://github.com/iliaal/fast_uuid

Happy to answer questions, especially from anyone running ramsey/uuid on a high-insert workload. I'd like to hear where the compat layer falls short of a real swap.

reddit.com

u/Ilia0001 — 2 months ago

▲ 40 r/laravel

fast_uuid: a near-drop-in ramsey/uuid replacement in pure C, 11-30x faster on the UUIDs Eloquent generates

If you use HasUuids or HasVersion7Uuids on your Eloquent models, every insert mints a UUID through ramsey/uuid, and ramsey/uuid calls random_bytes() once per UUID. That syscall is the bulk of the generation cost. On a write-heavy app, or a batch insert of a few thousand rows, it adds up to a slice of CPU that does nothing but produce identifiers.

I generate a lot of UUIDs, so I wrote a C extension to do it faster.

fast_uuid is a PHP extension (pure C, no C++) covering every RFC 9562 / 4122 version: 1, 2, 3, 4, 5, 6, 7, 8, plus nil and max. The object API mirrors ramsey/uuid under a FastUuid namespace, so for the common case migration is mostly a use swap. Two things make it fast:

Batched entropy. Instead of one random_bytes() per UUID, it pulls one getrandom() into an 8 KB per-thread buffer and amortizes it across ~500 v4s. The syscall stops dominating.
A SIMD hex formatter. 16 bytes to 32 hex chars in a handful of vector ops (SSSE3 on x86-64, NEON on ARM64, scalar fallback elsewhere). No build flags.

For UUIDv7 it carries sub-millisecond ordering (RFC 9562 6.2 Method 3), so same-millisecond v7s still sort in insert order, which is the whole point of a v7 primary key for index locality.

Benchmarks vs ramsey/uuid 4.9.2, PHP 8.4.22 NTS non-debug, best of 40 runs (million ops/sec, higher is better): v4 gen 19.5 vs 1.10, v7 gen 19.8 vs 0.66, parse 16.2 vs 3.18. That's ~11-18x on v4, ~18-30x on v7. One honest caveat: fast_uuid per-op time is ~50 ns, low enough that scheduler noise dominates a single run, so read its numbers as order-of-magnitude (±10% run-to-run). ramsey/uuid reproduces to within ~3%.

Before you swap, two notes. The FastUuid\Compat layer isn't on Packagist yet (install it as a Composer path repository for now), and a custom random or time generator intentionally routes off the C fast path, same as ramsey/uuid. There's also a uuid_v4_fast() using a non-crypto PRNG for non-security IDs only.

Install: pie install iliaal/fast_uuid

https://github.com/iliaal/fast_uuid

Happy to answer questions, especially from anyone running UUIDv7 primary keys at volume.

u/Ilia0001 — 2 months ago

▲ 30 r/PHP

phpser: a faster, HMAC-signed binary serializer for PHP cache workloads, benchmarked against igbinary

I've reached for igbinary on basically every PHP project I've shipped for the last decade. It's the obvious default for cache serialization. Two things about cache workloads kept nagging at me though, so I wrote phpser to see if a serializer built specifically for caches could do better.

The first is the read/write asymmetry. A cache decodes on every read and encodes once per write, easily 100:1 on a read-heavy cache, but igbinary (like most general serializers) balances the two sides. The second is trust: the bytes you decode often come from redis, memcached, or a cookie, any of which an attacker may be able to write to, and unserialize() on attacker-controlled input is one of PHP's oldest exploit primitives.

phpser is a C extension that goes after both. The wire format is designed around the reader (I borrowed the "make the reader do the least work" instinct from Rust's rkyv, though phpser is not zero-copy): a front-loaded string dictionary the decoder reuses by refcount instead of re-allocating, tagged scalar runs for packed numeric arrays, and pre-sized hashtables written in place. The encoder is fast too, with an O(1) pointer-hash string intern and plain objects serialized straight from their property slots.

Benchmarks vs igbinary (PHP 8.4 NTS release build, 1000 iters, median of 9 runs):

Shape	Size	Encode	Decode
packed_1k (range 0..999)	-65%	-70%	-75%
dto_1000 (Laravel queue shape)	-12%	-15%	-18%
rowset_1000 (mixed assoc)	+1%	-55%	+4%

It's not a clean sweep: mixed associative rowsets decode about 4% slower and run a few percent larger, because the front-loaded dictionary (the thing that makes everything else fast) doesn't pay off when few strings repeat. It's not streamable either, for the same reason.

On the security side there's an HMAC-SHA256 signed mode: phpser_serialize_signed($value, $key) and phpser_unserialize_signed($payload, $key). The signature is verified in constant time before any decoding happens, so a tampered or foreign-keyed payload returns null and never reaches the code that builds objects. There's also an allowed_classes option matching native unserialize().

Install is via PIE: pie install iliaal/phpser

Repo: https://github.com/iliaal/phpser Full writeup with the wire-format walkthrough and the complete benchmark table: https://ilia.ws/blog/phpser-a-fast-secure-binary-serializer-for-php-cache-workloads

I maintain php_excel and a few other PHP extensions; this one scratched a specific itch. Happy to answer questions, and I'd love feedback from anyone running heavy cache or queue traffic where decode time actually shows up in a profile.

u/Ilia0001 — 3 months ago

▲ 32 r/PHP

fastjson 0.3.0: drop-in faster ext/json for PHP, backed by yyjson (6× encode, 2.7× decode, 5× validate)

I maintain fastjson, a native PHP extension that drops in next to ext/json with a namespaced fastjson_* API and the same flag/error model. 0.3.0 just landed.

The problem. ext/json leaves a lot of performance on the table on both encode and decode paths. High-throughput PHP APIs spend a non-trivial fraction of CPU budget in JSON serialization. The two existing escape hatches are uncomfortable: simdjson_php is decode-only and not API-compatible, and rolling a custom validator is fragile.

The shape. fastjson exposes fastjson_encode/decode/validate behind a namespaced API that mirrors ext/json's. Backed by yyjson 0.12.0 (MIT). PHP 8.3 minimum; coexists with ext/json so adoption is opt-in per call site, not a repo-wide flag day.

Numbers. Full simdjson_php canonical 14.8MB corpus, 15 files, i9-13950HX, release builds of both PHP (8.6.0-dev) and fastjson:

Decode (stdClass): 602 MB/s vs ext/json 227 MB/s = 2.66×
Decode (assoc array): 628 MB/s vs ext/json 237 MB/s = 2.65×
Encode: 1,092 MB/s vs ext/json 180 MB/s = 6.06×
Validate: 1,352 MB/s vs ext/json 265 MB/s = 5.10×

Visual side-by-side (also includes ext/json (https://github.com/php/php-src/pull/17734) SIMD encode and simdjson_php on the same PHP build): https://iliaal.github.io/fastjson/baseline.html

Drop-in mechanics. fastjson_* signatures track ext/json. JSON_* flags and JSON_ERROR_* constants match byte-for-byte; fastjson_last_error mirrors json_last_error. Migration is search-and-replace.

Honest tradeoff. Decode and validate hold the yyjson doc in memory alongside results. Decode peak heap is ~1.7× ext/json's. Validate peak is ~101× ext/json's streaming validator (constant ~80 bytes), already 2.7× better than yyjson's stock read path thanks to a vendored patch. Encode is one-stage (direct write into smart_str), peak ~1.06× ext/json. If you are validate-heavy on huge inputs under tight memory_limit, the memory profile is a real consideration. For most callers the speedup wins.

What 0.3.0 does:

~36% speedup on object-heavy decode under JSON_INVALID_UTF8_IGNORE/SUBSTITUTE. A no-alloc UTF-8 validator scans each string and object key first; the sanitizer only runs on byte sequences that actually need replacement. Valid UTF-8 inputs no longer pay a per-string sanitize allocation and copy.
HEX flag rewrites (JSON_HEX_TAG/AMP/APOS/QUOT) now scan first and skip the rewrite entirely when no candidates exist. ~4× faster on the no-hit case (532 µs to 125 µs on 1k strings), ~13% regression on the all-hit case from the extra scan pass.
dw_emit_double checks the cheap range bound before calling floor(). Non-integer or out-of-range doubles in number-heavy arrays no longer pay libm per element.
Two correctness fixes: a use-after-free in fastjson_encode for PHP 8.4 objects whose property has a SET hook but no GET hook (engine's trivial-read fast path returns a borrowed pointer; the previous stash logic freed it), and a 32-bit zend_long overflow on the integer-valued-double shortcut that could emit INT32-saturated garbage for things like fastjson_encode(1e10).
ext/json parity for integer-valued doubles between 1e15 and 1e17. fastjson_encode(1e16) now emits "10000000000000000" instead of yyjson's "10000000000000000.0", matching json_encode.

Install via PIE:

pie install iliaal/fastjson

Repo: https://github.com/iliaal/fastjson

Open to feedback on flag coverage and edge cases, especially anyone running JSON_INVALID_UTF8_IGNORE or the HEX flags on hot paths.

reddit.com

u/Ilia0001 — 3 months ago

▲ 46 r/PHP

fastchart 0.2.0: native PHP charting extension with 19 chart types, plus Code 128 and QR codes

I maintain a handful of native PHP extensions. fastchart is the newest. 0.2.0 just landed.

The problem. PHP server-side charting is in rough shape. JpGraph hasn't seen meaningful work in years. pChart is abandoned. The common workaround is a Node or Python sidecar microservice that exists just to render PNGs. For OHLC plus indicator panes there isn't a serious PHP-native option at all.

Some history. In 2006 Rasmus and I shipped PECL/GDChart, a binding for the gdchart library. It died with its upstream in 2007. Since then I've built about six private PHP chart extensions, each solving exactly one need (a QR variant, OHLC for a dashboard, a couple of chart types). None shipped. fastchart is the consolidation.

What's in it:

19 chart classes: Line, Area, Bar, Scatter, Bubble, Pie, Stock, Radar, Polar, Surface, Contour, Gauge, Gantt, BoxPlot, Treemap, Funnel, Waterfall, Heatmap, LinearMeter
StockChart with 7 candle styles (CANDLE / BAR / DIAMOND / I_CAP / HOLLOW / VOLUME / VECTOR), SMA/EMA/WMA overlays, plus RSI / MACD / Bollinger Bands / Parabolic SAR / Stochastic / OBV indicator panes
A parallel Symbol family (new in 0.2.0): Code 128 (ISO/IEC 15417, auto subset switching, mod-103 checksum) and QR Code (ISO/IEC 18004, ECC L/M/Q/H, versions 1-40, vendored nayuki encoder)
Output to PNG, JPEG, WebP, AVIF, GIF
105 public methods, 86 phpt tests, PHP 8.3+ (NTS or ZTS), BSD 3-Clause

Install via PIE:

pie install iliaal/fastchart

Requires ext-gd (PHP's bundled GD extension); fastchart renders through gd.

Repo: https://github.com/iliaal/fastchart

Full writeup with the StockChart indicator stack and the composition pattern: https://ilia.ws/blog/fastchart-0-2-0-native-php-charts-barcodes-and-qr-codes-in-one-extension

Open to feedback on chart types worth adding next and on the StockChart indicator set.

u/Ilia0001 — 3 months ago

▲ 3 r/ClaudeCode

I built whetstone to stop Claude Code from declaring "done" before verifying anything

Six months of daily Claude Code use convinced me the agent doesn't lack capability. It lacks process. It can write a React component or trace a segfault, but it won't ask "did I verify this actually works?" before declaring victory, and it won't split a 400-line diff into reviewable chunks.

whetstone is a plugin that fixes that. 30 skills, 19 agents, 22 commands. Skills are compact instruction sets that load on keyword match, so the agent picks up the relevant discipline for the task it's actually running.

The debugging skill blocks a fix until the root cause is identified with file-and-line evidence two levels deep in the call chain. Reproduce first, one hypothesis at a time, escalate after three failed attempts instead of guessing forever. The code-review skill runs spec-compliance first, then quality, and dispatches parallel specialist agents (security, performance, database) when a diff crosses the size threshold. The writing skill keeps a banned-vocab list and a five-dimension rubric every piece of prose passes through.

The workflow is five commands. /ia-brainstorm interviews you to surface hidden requirements. /ia-plan turns that into atomic tasks with file paths. /ia-work executes with task tracking and verification gates. /ia-review runs the multi-agent review. /ia-compound captures what you solved as searchable docs the next session can find. Each works standalone.

https://github.com/iliaal/whetstone

reddit.com

u/Ilia0001 — 3 months ago

▲ 19 r/PHP

mdparser 0.3.0: native PHP CommonMark + GFM parser, 15-30× faster than pure-PHP

I posted this in r/laravel last week. u/equilni's reply:

> Looking at the repo, this looks like it could be used for plain PHP too. I suggest posting in that sub as well.

So here it is. Original thread: https://www.reddit.com/r/laravel/comments/1t84fu4/mdparser_030_native_php_commonmark_gfm_parser/

I build native PHP extensions when pure-PHP solutions become a bottleneck. mdparser is the markdown one. It wraps embedded cmark-gfm (CommonMark 0.31, all 652 spec examples pass) and ships as a single .so on Linux/macOS or .dll on Windows. PHP 8.3 minimum.

If your app renders markdown on every request, comment threads, docs, CMS pages, forum threads, transactional mail, pure-PHP parsers become a measurable fraction of request time. mdparser is for that hot path.

What's in it:

GFM extensions: tables, strikethrough, task lists, autolinks, tagfilter (XSS-safe HTML sanitization)
Smart punctuation, footnotes, safe mode, heading anchors, nofollow links
Three output formats from one parser: HTML, CommonMark XML, and a PHP AST (nested arrays). AST output is rare in PHP markdown libraries; useful if you want to walk the tree before rendering, or sanitize at the structural level instead of post-hoc on HTML.

Performance against the major pure-PHP parsers, on PHP 8.4 with each parser in its default configuration:

Parser	Small (200 B)	Medium (1.8 KB)	Large (200 KB)
mdparser	30,447 ops/s	5,697 ops/s	105 ops/s
Parsedown	1,651 ops/s (18x slower)	325 ops/s (17x)	6 ops/s (17x)
cebe/markdown (GFM)	1,350 ops/s (22x)	374 ops/s (15x)	6 ops/s (16x)
michelf (Markdown Extra)	1,006 ops/s (30x)	209 ops/s (27x)	5 ops/s (19x)

15-30× faster across the board, from short messages up to full 200 KB spec documents. league/commonmark is the closest competitor and has slightly different positioning (more extensions via opt-in); numbers and methodology in bench/README.md.

Install:

pie install iliaal/mdparser

API:

use MdParser\Parser;
$parser = new Parser();
$html = $parser-&gt;toHtml($markdown);
$ast  = $parser-&gt;toAst($markdown);

Blog post with the full benchmark methodology and comparison data: https://ilia.ws/blog/mdparser-a-native-commonmark-gfm-parser-for-php Repo: https://github.com/iliaal/mdparser

Happy to answer questions, especially about the AST output, the cmark-gfm postprocess interactions (heading-anchor positioning under raw HTML, nofollow-aware HTML scanning), or anything PHP-extension-side.

reddit.com

u/Ilia0001 — 3 months ago

▲ 29 r/laravel

mdparser 0.3.0: native PHP CommonMark + GFM parser, 15-30× faster than pure-PHP for high-volume Laravel rendering

I build native PHP extensions when pure-PHP solutions become a bottleneck. mdparser is the markdown one.

If your Laravel app renders markdown on every page load (comment threads, mailables, Filament fields, content pages) pure-PHP parsers like league/commonmark and Parsedown become a measurable share of request time. mdparser is a C extension that parses CommonMark and GFM 15-30× faster on the same documents. league/commonmark is a fine default for most apps; the pain shows up when markdown rendering is on the hot path.

What it does:

GFM extensions: tables, strikethrough, task lists, autolinks, tagfilter (XSS-safe HTML sanitization)
Smart punctuation, footnotes, safe mode
Output as HTML, XML, or PHP AST (the AST output is rare in markdown libraries; useful if you want to walk the tree before rendering)

Where it slots into a Laravel codebase:

Mailable rendering. The path that ships with Laravel goes through league/commonmark under the hood, so swapping in mdparser for high-volume transactional mail is a one-line change in the renderer binding.
Filament markdown fields, rendered on the backend.
Forum or comment rendering middleware.
Documentation or static page generation.

Install:

pie install iliaal/mdparser

API:

$parser = new MarkdownParser();
$html = $parser-&gt;toHtml($markdown);
$ast  = $parser-&gt;toAst($markdown);

Blog post with the full benchmark methodology and comparison data: https://ilia.ws/blog/mdparser-a-native-commonmark-gfm-parser-for-php

Repo: https://github.com/iliaal/mdparser

Happy to answer questions about Laravel-specific integration, mailables especially.

reddit.com

u/Ilia0001 — 3 months ago

▲ 127 r/PHP

I've maintained php_excel since 2008. 2.0 shipped in April as the first ground-up rewrite, and 2.0.1 just landed on May 3.

The problem it solves: PhpSpreadsheet builds the whole DOM in PHP memory. On a 50K-row spreadsheet you're looking at ~790 MB resident before you've called save(). In a 128 MB FPM pool that means OOM on anything past trivial. OpenSpout streams, but it can't write conditional formatting, formulas, rich text, or .xls.

php_excel wraps LibXL through a native C extension. LibXL (libxl.com) is a commercial C++ library by Andrew Karasyov, not mine; you acquire it separately. php_excel is the PHP binding.

Quick comparison. PhpSpreadsheet is the most feature-complete pure-PHP option. OpenSpout fills a streaming niche. php_excel is the C-extension answer for when you need both speed and full Excel features.

What 2.0 changed:

PHP 8.3 / 8.4 / 8.5 / master, dropped older versions
LibXL 4.6.0+, newer features gated at compile time
12 classes (6 new: ExcelRichString, ExcelFormControl, ExcelConditionalFormat, ExcelConditionalFormatting, ExcelCoreProperties, ExcelTable)
399 typed parameters, 277 typed return values, full arginfo coverage from a stub-driven build
Installable via PIE

Benchmarks against PhpSpreadsheet 5.5.0, PHP 8.4.19 NTS:

Rows	Cells	php_excel	PhpSpreadsheet	Speed
1,000	20K	0.05s / 85 MB	0.45s / 162 MB	10×
10,000	200K	0.55s / 153 MB	4.59s / 282 MB	9×
50,000	1M	2.72s / 508 MB	24.7s / 790 MB	9×
100,000	2M	5.37s / 908 MB	51.1s / 1,415 MB	10×

Read perf is similar: 8-9× faster than PhpSpreadsheet, 3× faster than OpenSpout (with proportional memory trade-off vs OpenSpout's flat 130 MB).

2.0.1 is a hardening pass: extensive error checking and input validation across the C/PHP boundary.

Install:

pie install iliaal/php-excel --with-libxl-incdir=/path/to/libxl/include_c --with-libxl-libdir=/path/to/libxl/lib

Full writeup with methodology: https://ilia.ws/blog/php-excel-2-0-the-c-extension-for-excel-that-php-should-have-had-all-along

Repo: https://github.com/iliaal/php_excel

reddit.com

u/Ilia0001 — 4 months ago

▲ 29 r/PHP

Six days ago I tagged 0.6.0 of php_clickhouse, a soft fork of the SeasClick extension (which stopped accepting PRs in 2020). Three releases later (0.7.0, 0.8.0, 0.8.1) I'm calling the extension stable.

The PHP ClickHouse ecosystem has been split between SeasClick (native binary protocol but stalled, no modern types, no ZTS, no TLS in the maintained fork) and HTTP clients like smi2/phpClickHouse (active but ~30-40% slower at high throughput). This fork picks up the native-protocol path with the official clickhouse-cpp v2.6.1 client and brings the modern type surface back.

What landed in the quality cycle:

0.7.0 closed the API gap with smi2/phpClickHouse. Per-call settings, server-side typed parameters via {name:Type} placeholders, progress callback, getStatistics() (rows/bytes/elapsed_ms), structured ClickHouseException with server_code / query_id, insertAssoc(), SQL helpers (databaseSize, showTables, tableSize, etc.), sub-second timeouts. Adopting the native client no longer costs ergonomics.

0.8.0 moved per-Client state from seven file-scope std::map banks onto the zend_object itself. Unblocks ZTS (RoadRunner / FrankenPHP / Swoole / php-pm now work), plugs leaks on script bailout, fixes a refcount bug on the progress callback. Adds streaming reads (selectStream() returns an Iterator + Countable, selectStreamCallback() for unbounded streams), Geo types (Point / Ring / Polygon / MultiPolygon), LowCardinality(Nullable(T)), and Map(K, V) over the full scalar matrix. Pre-built binaries for Linux glibc (x86_64 + arm64) and macOS (x86_64 + arm64) via PIE.

0.8.1 hardened the insert path. The connection now resets on every server-side rejection point (BeginInsert, SendInsertBlock, EndInsert) so a thrown insert no longer wedges the handle with "cannot execute query while inserting". Insert builds native columns one at a time directly from row-major input: peak intermediate PHP memory drops from N_rows × N_cols zvals to one column. Strict full-consumption parsers across Map, narrow-int, Int128 / UInt128, geo, DateTime64 reject coercion-to-zero on bad input. 23 new PHPTs covering all of the above.

One side effect of the new ASan job: it caught a latent UB in clickhouse-cpp's empty-string-view memcpy path. Submitted upstream and merged as ClickHouse/clickhouse-cpp#489 on April 27.

Install on supported platforms (Linux glibc + macOS, NTS, PHP 8.4 / 8.5):

pie install iliaal/php_clickhouse

TLS variant builds from source: pie install iliaal/php_clickhouse --enable-clickhouse-openssl.

Blog post with the full breakdown: https://ilia.ws/blog/php-clickhouse-0-8-1-three-releases-later-stable Repo: https://github.com/iliaal/php_clickhouse

Happy to take questions, especially from anyone running ClickHouse-from-PHP at production volume.

reddit.com

u/Ilia0001 — 4 months ago

▲ 7 r/pinescript+1 crossposts

Pine Script editing is mostly a manual loop in TradingView's editor: write, compile, read errors, fix, repeat. AI agents can help if you copy-paste the script and errors into Claude or Cursor and paste the fix back, but the agent can't see what the indicator actually plots: labels, lines, boxes, tables, plotshape markers. So visual debugging stays manual.

I built tradingview-mcp to put Pine Script in the agent's hands directly. It's an MCP server (96 tools across the TV surface) plus a tv CLI; both drive your local TradingView Desktop over Chrome DevTools Protocol. The Pine-specific tools:

pine_check: server-side compile without putting the script on a chart. Useful for CI-style verification or letting the agent validate a draft before adding it.
pine_analyze: offline static analysis (catches typos, unused vars, deprecated patterns) before you compile.
pine_smart_compile: auto-detects whether to add or update, returns elapsed_ms.
pine_save_as, pine_rename, pine_version_history, pine_delete, pine_switch_script: full lifecycle, no editor clicks.
data_get_pine_lines / _labels / _tables / _boxes / _shapes: the agent can read what the indicator actually drew. Horizontal price levels, text annotations, table cells, price zones, plotshape markers. Deduplicates and caps output by default; opt into raw via verbose.

The visual-output readers are the part I keep using most. Agent writes an indicator, compiles it, reads the labels back, decides whether the logic is right.

A few release details: Pine Editor open + symbolInfo fallbacks for TV Desktop 3.1.0 (compile/deploy buttons matched by title attribute). pine_set_source no longer hangs on large scripts. 338 offline tests cover the Pine tooling, multi-timeframe alignment, replay, and CLI routing. The upstream ui_evaluate tool (arbitrary JS in your authenticated TV session) was removed from the surface; everything else is gated through specific tool boundaries.

Install: clone the repo, npm install, add to ~/.claude/.mcp.json, launch TradingView with --remote-debugging-port=9222. README has the paste-into-Claude-Code one-liner.

Repo: https://github.com/iliaal/tradingview-mcp

Happy to answer questions, especially from anyone running heavy indicators with many lines / labels: the dedup defaults are calibrated to my workflow and may need tuning for others.

u/Ilia0001 — 3 months ago

▲ 20 r/PHP

A while back I pushed a small PECL extension that wrapped libstatgrab and exposed CPU, memory, disk I/O, network, and process statistics to PHP. It sat untouched for most of the PHP 5/7 era and stopped building cleanly on PHP 8 a few years back. I shipped 2.2 today, a full modernization of the binding for PHP 8.0 through 8.5 against libstatgrab 0.92+.

The reason to revive it: nothing on the PHP side has replaced it. If you need system stats from PHP, you are typically choosing between three options.

Shell out to w, vmstat, df, ps. The output format drifts between OS releases, and you end up writing a per-tool parser.
Parse /proc by hand. Linux-only, every file (meminfo, loadavg, diskstats, net/dev) has its own format and edge cases.
Run a separate stats daemon and hit it over a socket. Adds a daemon to deploy and keep running.

libstatgrab itself is the right primitive: a cross-platform C library that handles /proc on Linux, kvm on FreeBSD, and the Mach host_* APIs on macOS, and exposes one typed surface. It just needed a PHP binding that worked on a current interpreter.

The 2005 procedural API is preserved (sg_cpu_percent_usage, sg_memory_stats, etc.) for drop-in compatibility, with a new OO surface (Statgrab::cpu(), ::memory(), ::processes()) on top.

While running ASan on the new test suite I caught a memory leak in libstatgrab's shutdown path. Patch submitted upstream; pending review. The repo carries a vendored libstatgrab 0.92.1 with the local fix in the meantime. Build with --with-statgrab=bundled to get a single .so with no runtime dependency on libstatgrab.so. Useful in any deployment where you don't want to require libstatgrab as a system package.

Install:

pie install iliaal/statgrab

Or pecl install statgrab if you are still on the legacy installer. Source build and the bundled-libstatgrab path are in the README.

Repo: https://github.com/iliaal/statgrab Full write-up: https://ilia.ws/blog/its-alive-statgrab-returns-after-20-years

reddit.com

u/Ilia0001 — 4 months ago