▲ 3 r/ETL

Help with Extract -> Load process logic in a personal project

Hello! I'm working on a small data project using a NASA API with Python. The idea is to extract the orbital elements and close approach (asteroids, comets, etc) objects (and there orbital elements) of each planet in order to perform some physics simulations with the data.

I was planning to use a Cloudflare R2 bucket to store the raw API responses, then transform them into a PostgreSQL database, and use FastAPI to consume the data.

I'm not sure how to perform the extraction and loading processes. In theory, each planet will follow this path:

  1. Fetch the orbit elements for the own planet (use API 1);

  2. Fetch the close approach object (use API 2);

  3. Fetch the orbit elements for each of the close approach objects (use API 1).

I use two APIs: one for the orbital elements and one for the close approaches. Should I fetch all the data, store it in memory, and then send it to the bucket? Or would it be better to load it right away with each API call? So, when fetching the close approach objects' orbit elements, get the JSON from the bucket and then use the first API and store that raw response in the bucket

reddit.com
u/xMexicanPizza — 14 days ago

Menos sueldo pero mejor empresa y proyecto.

Acabo de egresar como Ingeniero Fisico en Mayo, pero me enfoque mas en data science/deep learning/machine learning en la carrera. Estoy trabajando en mi primer empleo estoy aqui desde enero, empece como practicante por mis practicas profesionales pero me contrataron de tiempo completo hace 1 mes. Estoy trabajando en una consultora de Odoo, pero trabajo en un proyecto aparte, basicamente hago scripts con APIs en Python para integrar a unos modulos de Odoo que utilizadan para migrar la base de datos de un cliente, yo no toco nada de Odoo, unicamente los scripts, pero siento que estoy estancado, realmente las tareas que me dan son algo basicas, y cuando no hay, me ponen a basicamente llenar Excel con unas investigaciones que se requiere, basicamente buscar datos en google y llenar el excel, ya me aburrio sinceramente. Gano 10k brutos, pero me dan extras bonos de proyecto/rendimiento, que suman hasta 2k-4k a veces.

Me postule a una vacante en una compañia mas grande en mi ciudad, que manejan distintas franquicias de comida rapida gringa. Buscan diseñar, entrenar e implementar modelos predictivos para sus metricas financieras apenas andan empezando con el proyecto, la verdad se ve interesante, pero creo que pagan menos de lo que ando ganando ahora o al menos eso parece.

Me arriesgo a ganar menos pero con la premicia de estar trabajando mas en el area que me interesa y en trabajar en una empresa mas grande? O me quedo donde estoy por el salario

reddit.com
u/xMexicanPizza — 2 months ago

Pixel 8A vs 9A, $180 USD difference

My phone broke, and I was thinking about buying my first Pixel. I have two options: a new Pixel 8A for $340 (OpenBox brand new) or a Pixel 9A for $520 (that’s the cheapest one I’ve found in my country). Is it worth spending that extra $180?

I want a small phone, so the fact that the 8A has a smaller screen is a plus for me, hahaha, but I don’t know if it’s a good choice in 2026. I’m not a demanding user—I don’t play games or anything like that. I just use it for basic stuff like social media, WhatsApp, YouTube, and browsing.

reddit.com
u/xMexicanPizza — 5 months ago

PAW3395SE vs PAW3395, Will I notice any difference in CS2?

I have a VXE R1 with the PAW3395 sensor, but a few days ago I dropped it, the shell cracked a little, and now the left click feels a bit stiff. While looking for options to replace the mouse, I saw that VXE released the Dragonfly V3 Pro mouse, which is exactly the same as the G305—a mouse I used for many years—as well as its wired version, the G203. Basically, I’ve been using that mouse shape since 2018 until I bought the R1 a few months ago, but this one comes with the 3395SE, which I understand is worse than the standard 3395. I only play CS2 at 144 Hz; will the difference between the sensors be noticeable? Do you think I should go for another mouse with a better sensor? it cost me $20, including shipping to my country, so it was really cheap.

https://preview.redd.it/p8of2ofcjb1h1.png?width=361&format=png&auto=webp&s=9979c5c44bc509b0a26e32640fbafb682adce593

reddit.com
u/xMexicanPizza — 5 months ago