r/mongodb

▲ 23 r/mongodb

Studio 3T Told Me I Couldn't Build a Better Tool — The Jobless Grad That Built VisuaLeaf

Last May, at a .locals event in London, I had an interesting encounter with someone from Studio 3T.

I had flown all the way from California to London for a trip and decided to attend the event while I was there. After I introduced myself and told them I was building VisuaLeaf, and... one person told me I wouldn't be able to build a better tool than theirs. So I took it personally. I was a fresh grad with no job, and VisuaLeaf was all I was working on, so I decided to prove them wrong.

Ever since I posted VisuaLeaf here in 2024 and it became the 2nd highest all-time post, I've watched this subreddit get flooded with people trying to build their own MongoDB tools, especially with AI.

Every week there's another MongoDB GUI that someone built in a weekend, calls it production-ready, and posts it here. It's hard to get excited about another tool that does the same basic things as everything else, especially when most of them still miss what actually matters: a smooth experience. A lot of them crash or lag when the data isn't even that big. I don't think a couple of prompts can replace thousands of hours of attention to detail, testing, and optimization. That's the experience I set out to build.

This is now my 2 year journey of building the best MongoDB GUI out there.

Honestly, too much has changed since the last time I posted here, and I think, for now, VisuaLeaf has all the features I really wanted in the app. It’s also stable enough that I’m now seeing people from actual companies using it daily, which caught me by surprise haha. I’m really happy that more people are discovering VisuaLeaf every day and that hundreds of people are actively using it every month.

Over the past two years, I spent an entire year optimizing a table (yes...), built a complex query builder, created a visual schema designer that can materialize MongoDB collections, added mongosync, and even built our own multithreaded dump utility, which we benchmarked against the shell's mongodump and found to be faster when exporting whole databases or multiple collections.

It may not sound like much but, the table is probably the thing I'm most proud of, even though it's literally just a TABLE.

I spent a LITERAL year optimizing it because I wanted it to feel as smooth as possible while working with large datasets. I basically built my own AG Grid from scratch because the libraries I tried didn't have the features I wanted like expanding objects or arrays as new columns.

It was one of the hardest optimization journeys I made but I’m really happy how it turned out because it’s literally one of the smoothest tables out there that is BUILT to visualize data for databases. The tree view was a lot easier since it was just 1 dimensional but still had its own tricks to expand thousands of nodes at the same time instantly (I also used the virtual scrolling engine I made for the table view to the tree view and made the tree view scroll smoother than official Angular virtual scrolling libraries). And during all that time, I focused on making the data retrieval to be as efficient as possible so much so that it was the fastest GUI out there for loading data. 

After all that, I genuinely don’t think there’s objectively a better MongoDB GUI out there, but maybe that’s just me haha.

I’m coming for you, 3T ;) (I’ll remember you telling me that I wouldn’t be able to make a better tool hehe- for the full story you can dm me lol)

If you want to check out all the changes, feel free to check it out here: visualeaf.com
And if you want to help me on my journey with VisuaLeaf feel free to join the discord : we could talk about bugs, features that you guys want, etc: https://discord.com/invite/TR6J56Y3rF

u/Fun-Chicken6946 — 13 days ago

Please help a junior engineer out!!!!!

I am given a task by my senior engineer, and I am at my wit's end at how to do it. I tried asking AI as much as I can without putting in my codebase, searching it up, looking at documentation until 4am yesterday. I am still stuck.

Suppose I have model X, which holds an array of model Y. X and Y both have status fields, but they are different. Y.status refers to that individual item's status, for example, accepted, cancelled, pending, etc. X.status refers to the overall status of all it's Y array items, it depends on their values. Usually, for a single X update, there is a function that iterates through that X's Y children, and it calls the .save() method. The feature I am tasked with implementing is

1: Making changes to a large number of X documents' Y children based on some criteria
2: Recalculating all the affected X's overall status since its children Y changed.

At first, my solution was to iterate over every X, call the function to calculate its new overall status, and use the .save method to update X's overall status. I see now this doesn't work at production scale. Promise.all() on potentially a large number of X documents may cause stack overflow, and it is a lot of write calls to the database which is not ideal.

My senior engineer told me to call that function inside a update many, but after the meeting I looked it up, and I am pretty sure mongo can't just call functions defined on the node server, and copy pasting the function again as a $function in a aggregation pipeline is also bad for performance.

There are a lot of if else branches in the function needed to calculate X's overall order status. I considered a bulk write with a lot of updateMany calls looking for a specific condition, but that would easily be over 10 updateMany's in the bulkwrite.

I am really stressed about this. Would love everyone's engineering input.

reddit.com
u/Mango__DB — 14 days ago
▲ 13 r/mongodb+1 crossposts

NEW: Slow query email reports

Hey everyone! I’m Shameek, I’m on the Engineering team at MongoDB. We wanted to make it easier for MongoDB developers to spot queries that might be slow. We started sending email reports to users that capture their top slowest queries in the past week, as well as efficiency stats of those queries. You can customize these emails and unsubscribe at any time.

I’m curious, do people think emails like these would help them? 

  • Is there anything else you'd want MongoDB to analyze and include in the weekly summary for your slow queries?
  • How are people currently finding their slow queries?

If anyone has any suggestions to improve these, drop them in the thread below! We'll be taking your feedback and suggestions back to the team 😊

u/ShameekDB — 14 days ago