Is AI actually replacing traditional methodologies in Scientometrics? (and why we still have a long way to go)

Hi everyone,

I’ve recently completed a data curation and comparative study analyzing the adoption of Artificial Intelligence, Machine Learning, and LLMs in scientometric research methodologies.

I analyzed a corpus of 706 papers from two different venues:

  1. Quantitative Science Studies (QSS - Volumes 1 to 6): representing the international, flagship journal level.
  2. Brazilian Meeting on Bibliometrics and Scientometrics (EBBC - 2020, 2022, 2024): representing the Brazilian national state-of-the-art.

We filtered the papers to identify those that did not just study AI as a topic, but actively incorporated AI/ML/LLMs as the core methodological motor (replacing or enriching traditional statistics, manual curation, or classical text mining).

Here are some of the numbers we found:

  • Overall AI Adoption Rate:
    • QSS (International)20.46% (80 out of 391 papers) used AI in their methodology.
    • EBBC (Brazil)3.49% (11 out of 315 papers) used AI.
  • The Temporal Spike:
    • Both venues saw a significant jump in 2024/2025. In EBBC, adoption went from a flat ~2% to over 5% in 2024. In QSS, it peaked at 24.59% in 2025. This is clearly the "ChatGPT effect" acting as a technological shortcut for researchers.
  • The Methodological Focus:
    • EBBC: Almost entirely concentrated in Natural Language Processing / Text Curation (90.9%)—mostly using LLMs out-of-the-box for text classification.
    • QSS: Highly diversified, with Graph Neural Networks / Network Embeddings (41.2%) and predictive modeling leading the way.

The Core Observation (My CS / Industry Perspective)

Coming from a Computer Science background and having worked in the tech industry, these results left me with a mixed feeling:

  1. The Brazilian scene (EBBC) is still very raw. Most applications are extremely basic using OpenAI APIs or web interfaces to perform simple tasks that could often be done more robustly with deterministic scripts, or using prompt engineering in a very naive way without evaluating bias, token efficiency, or reproducibility.
  2. The international scene (QSS) is much more mature, but it still underperforms. Even at the highest level of academic publishing, I see researchers treating LLMs as magical black boxes. There is a noticeable lack of software engineering best practices. We see sub-optimal API pipelines, lack of structured outputs (like JSON schemas), no system evaluation frameworks, and a general lack of computational efficiency. It feels like many academics are lagging behind the industry standards of how to properly deploy, prompt, and scale AI-driven workflows.

It seems that while AI is indeed starting to replace or augment traditional bibliometric methodologies, the usability and efficiency of these tools among academics is still a massive bottleneck.

I’d love to hear your thoughts and experiences:

  • How are you integrating AI/LLMs into your research pipelines?
  • Have you noticed this gap between academic AI methodology and industry/CS standards?
  • How can we, as a community, bridge this gap and help researchers from non-CS backgrounds use these advanced tools more efficiently?

Looking forward to the discussion!

reddit.com
u/Solid-Play-458 — 4 days ago
▲ 49 r/design_critiques+1 crossposts

Logo Concept for My New App

Currently developing a side project called Paperback, a minimalist e-book reader focused on co-reading.

I don't have experience with this kind of thing, but so far I've been enjoying it. What do you all think?

Also, for anyone who wants to try it out, it's currently in beta: https://paperback.fly.dev/

u/Solid-Play-458 — 8 days ago

Giving up on my open-source project because of gatekeepers.

Hey everyone,

I'm honestly feeling pretty discouraged and I'm seriously considering archiving one of my latest open-source projects.

I built an open-source GitHub Action called tabnews-release-publisher. The idea is simple: when you publish a GitHub release, want to share an announcement, ask for feedback, or start a discussion, the Action can automatically publish that content to TabNews (a brazilian forum). For international projects, it can also use an LLM (Gemini, OpenAI, or Claude) to translate and adapt the text to Portuguese.

The goal wasn't to generate content, but to automate something many maintainers already do manually: adapting GitHub posts, formatting them for another platform, and publishing them again. I also planned to expand it by automatically extracting repository information, generating forum-friendly Markdown, and adding release metadata to make posts more useful.

I wasn't trying to invent a new workflow. Similar integrations already exist in many developer communities and forums. I simply wanted to bring that kind of automation to the Brazilian ecosystem.

However, when I shared the project, the reaction was overwhelmingly negative. Instead of discussing the idea or suggesting improvements, most of the feedback was dismissive and accompanied by downvotes. That honestly made me question whether it's worth continuing.

So I'd really appreciate some outside opinions:

  • Does this sound like a useful workflow or something you'd avoid?
  • Would you ever use a tool like this for one of your own repositories?
  • Have you ever considered abandoning a project because the initial feedback was more hostile than constructive?

I'm not looking for validation. If you think the idea has fundamental flaws, I'd genuinely like to hear them. I just want perspectives from people outside my local community before deciding whether to continue or archive the project.

reddit.com
u/Solid-Play-458 — 9 days ago
▲ 6 r/BibliometricScience+3 crossposts

FREE: A public API & dataset for Bibliometrics and Scientometrics metadata ( Brazil )

I wanted to share a project I've been working on called EBBC OpenData, which is a public API and dataset designed to promote Open Science and support bibliometric, scientometric, and informetric analyses. You can find the full project and source code in the repository at https://github.com/GabrielBaiano/EBBC-OpenData

This project provides structured metadata from the publications of the Encontro Brasileiro de Bibliometria e Cientometria (EBBC), which is one of the main events on metric studies of information in Brazil. Through this API and dataset, you can easily query detailed information about authors and their academic networks, articles and papers (including titles, abstracts, and publication years), institutions associated with the research, keywords, thematic trends, as well as references and citations.

The core metadata and documentation are currently being organized, and I am actively working on translating the API documentation and dataset fields into English and Spanish to make the project fully accessible to the global research community.

Since this is an ongoing project, I would highly appreciate your thoughts and feedback. I am especially interested in knowing what features or endpoints would make this more useful for your research, any suggestions you might have regarding the data structure or documentation, and any general tips on best practices for open-data APIs. Please feel free to check out the GitHub repository, open an issue, or leave a comment below. Thanks for your support!

u/Solid-Play-458 — 10 days ago

Any Tips on Getting a Playdate in Brazil Without Breaking the Bank?

Hi everyone!

I'm a game developer from Brazil currently working on a Playdate game with a small team. We're planning to release a demo soon, and I'd really love to have a Playdate of my own for testing and daily development.

I've had the chance to use one before by borrowing it from someone and also during a few events, so I already know how great the device is. The problem is that importing one to Brazil is extremely expensive due to shipping and import taxes.

Does anyone know of a more affordable way to get a Playdate in Brazil? Maybe a used marketplace, a reseller, or someone in the community looking to sell theirs?

Any tips or recommendations would be greatly appreciated. Thanks!

reddit.com
u/Solid-Play-458 — 12 days ago

I built an interactive Playdate kiosk/panel for a small gaming event in Brazil

Hey everyone!

I wanted to share a small project I made for a local gaming event in Brazil.

I'm a big fan of the Playdate and wanted people to be able to learn about it and interact with something inspired by the console, even if they had never seen one before. So I created an interactive touchscreen panel that was displayed at my booth during the event.

The link ! playdate-picad.vercel.app

Git Repo > https://github.com/GabrielBaiano/--Playdate-landing

u/Solid-Play-458 — 1 month ago