r/highfreqtrading

Hey everyone,

I’m looking at data -> research workflows for Mid-Frequency Trading (MFT) ML pipelines using Python (Ray, Polars).

A common industry trend is using table formats like Apache Iceberg or Delta Lake on object storage (S3). However, if you already enforce a highly static, predictable directory layout (e.g., ⁠equities/exch=.../year=.../⁠ with <50 optimized Parquet files per leaf), I'm struggling to see the value.

In a high-performance research environment, it seems far more practical to treat object storage strictly as a cold source of truth, sync the required historical partitions directly onto the compute nodes' local NVMe scratch disks, and run active Python training loops entirely on local NVMe.

If you are caching a predictable folder structure down to local NVMe anyway, does an object-store table format buy us anything substantial, or is it just added complexity?

For those working on Quant Platform or QR Infrastructure teams: Do you actually query Iceberg/Delta tables directly from cloud storage during active research, or do you use the "Cloud Archive -> Local NVMe Hot Compute" pattern?

Thanks!

Hi everyone,

I have an upcoming interview for a C++ Developer position at Trexquant and wanted to ask if anyone here has gone through their interview process recently.

I'd really appreciate any insights on:

• What technical topics should I focus on?
• How deep do they go into modern C++ (C++17/C++20), STL, templates, memory management, multithreading, and concurrency?
• Are there any data structures and algorithms or competitive programming style questions?
• What kind of coding problems can I expect?
• How many interview rounds are there, and what does each round typically cover?

If you've interviewed there recently, sharing your experience or any sample questions would be extremely helpful.

Thanks in advance!

I am building a market making engine for illiquid markets (from few to say 50 trades an hour). It shows positive returns, after testing for the last 2 weeks. And I have a few questions I was thinking:

All the MM theory (such as Avellaneda Stoikov framework) are targeting high volume, low spread markets. Any good papers/frameworks built for illiquid market? What is the main difference?

From my point of view there are 3 types of events: small single market trade, huge single market trade, and huge informed (not single) trade. The last one is toxic, the first two are good ones. I am failing to distinguish them, and usually it's already too late.

Earlier I experimented with hand crafted quoting algorithms, but apparently the most effective one is simply quote at the top (with some exceptions like filters for OB imbalance, skip small levels etc) and eat the spread. I think the right approach is to quote at multiple levels, so I will earn more on rare events - huge trade arrivals. I failed to find some standard way to do so, or the way that will at least beat single level quoting.

Any ways to avoid toxic orders without losing much volume?

How to distinguish uninformed huge trade vs informed huge trade? Should I immediately sell if the price dropped or should I hope it will return back (uninformed single trade)?

Recently I'm working on a vector database from scratch in C++ to better understand how modern vector database works. I just finished benchmarking against SIFT1M dataset (1M vectors, 128 dim) and wanted to share the performance and design choices and get feedback.

Core architecture details-

• Instead of vector<vector<float>> , used 64 Byte aligned allocator to maximise L1/L2 cache locality.
• implemented AVX2 FMA SIMD intrinsics for distance computation (L2, dot product, cosine).
• Implemented HNSW graph for Aproximate nearest neighbor, elminating O(n) linear search.

SIFT1M Benchmark Results (single threaded, Intel i5-13420H, compiler flags ( -O3 -march=native -ffast-math)

• Recall@10: 99.30%
• Recall@100: 98.26%
• Throughput: 2,215 queries/sec
• Tail Latency (p99): 654 microseconds
• Build time: ~13 min.

To establish a baseline, I benchmarked my implementation head-to-head against hnswlib on the same machine using identical parameters (M=32, ef_construction=400, ef_search=200).

• Throughput: 2,215 QPS vs hnswlib's 2,745 QPS (~19% slower).
• Recall@10: 99.30% vs hnswlib's 99.88%.

Right now, my implementation optimised for single thread traversal and I prioritized read-throughput first using a global shared_mutex. My next major task is concurrent writes. I'm looking into fine-grained spinlocks per node, but I want to ensure strict lock ordering to avoid deadlocks and TOCTOU conditions.

For concurrent insertions into HNSW, would you lean toward per-node spinlocks, lock striping, or another approach?

github repo link -> https://github.com/randomfunction/vector_database/tree/main