u/Helpful-Series132

[removed]

As of now this base model is only capable of autocompleting in the style of an arXiv abstract, and our prototype q/a model can answer seen questions at a fairly decent rate. We believe that we can potentially create this agent by distilling new models doing deep research tasks.

We are aware that our tiny dataset of 3m arxiv abstracts isnt enough so we are looking for more information related to science, physics and technology to develop a deeper understanding on these topics. The expectation of achieving the goal is somewhere around 200m-500m parameters due to models like Liquid AI, LFM-230m & 700m being able to achieve similar goals. The new wave of tiny models has inspired us to go even smaller by focusing on one task which is research.

We are aiming to make this model perform autonomously using web search, CRUD file documentation & communication through an app designed to run locally on your device to update the user.

If you would like to contribute please send links of any datasets for fine-tuning please. If you write datasets please reach out we are also looking for advice from anyone with experience for writing examples. It is a challenge to shape autonomous behavior without a human input especially for a small model, but we believe that there is a way to achieve this through repetition/information density.

Also please share your thoughts, we are new to developing models from scratch and we have a lot of experiments that you can run yourself using our notebooks on kaggle. Our smallest experiment i a 2m parameter model which has a dataset written 100% by hand. It only talks about itself, but it able to understand and carry a short conversation if you engage with how it prompts you back.

Our long term goal is to be the company that hires hundreds of authors to write the dataset of a model 100% from scratch but first we must prove we can achieve our hypothesis through curation. This dedication and investment in time is all experimental research from a small team to find new discoveries and capabilities in small models.

Please believe in us, our goal is to achieve what nobody has done before & we try our best to create these formulas of data to give to you for free on a device that anyone can run locally

This is my own speculations i want to hear your opinion in the comments .. but if Zhipu makes a language model thats around 30b parameters this can potentially pop the AI bubble .. the bubble popping isnt that we will stop using AI .. its the fact we wont need to pay for it to do the goals we're aiming for & i think this is what closed source companies dont want.

Were getting a lot closer to free local models than we are to AGI & the smaller they get the more accessible they are. Everyone isnt trying to make a super system some people just want websites & honetly .. most people in the world just ask AI questions nobodys paying for a subscription besides software developers and a few creatives/business owners.

When a local model can perform the same tasks its a wrap .. will you still pay $200 a month if a small model can handle all of the work just a lil bit lower?

ive never paid more than $20 and ive achieved the goals ive aimed for so i dont know about you but im 100% fine with a small model

im a director that spent 12-18 hour a day editing videos .. once i started getting familiar with computer science and deep learning, ai looked far less impressive than it initially did

i made a lot of websites and apps i like how my ui looks. im not much of a graphic designer which is basically web design in my eyes, i think the ui i make got a lil taste to it (from my touch .. the templates be weak as fuck) .. i told Claude what to do bro he did it for me lol .. ive made some that were better in the past but this is the most recent one .. i needed the liquid glass effect, apple honestly did half the work they made it easy to build with

My issue with AI isnt that it cant do something. It simply doesn't do it good enough for me to be impressed to pay anything beyond $20 & honestly .. i can do better on my own without it

its like the ai industry scares people into thinking the opposite .. as a director/video editor for 10 years working on projects from music vids to a super bowl commercial, i can tell you right now as a product this is not even really that valuable like its promoted to be because it can actually hold you back & ill explain how (ive developed many skills, editing, color-grading, lighting, scoring, audio engineering, music production, directing, filming, using gimbals, drawing, painting, playing instruments, keyboard, guitar, skateboarding, i learned real life skills consistently across my entire life and i am competitive in each one)

ive built all types of agents, i design language model architecture, ive made my own models from 2m - 43m parameters from scratch .. i learned a lil bit of deep learning about neural nets to be able to make my own algorithm of token prediction (all of these are public on kaggle open source by the way) its fun bro but im competitive .. i shot music videos for some of the top artists in rap, i shot reality shows, short film .. people love my work .. i understand the art of a skill & ai just isnt that good to me. im sorry if i feel the other way around but idk what to tell you, you cant convince me this tool is amazing i think its worth about $10-20 month max

i think if your a straight tech person they made you have this illusion that its better than you or that you cant catch up with it .. since i was brand new with prior success in polishing products and creating art for clients i dont see it the same way

i been making pretty good UI (if u ask me) for a while it was tedious prompting before fable & 5.6 but now they all have a nice handful of templates to start off with .. it just feels like i wasted time when i could have spent months learning UI thats far more creative and unique.

i enrolled in college to learn CS just because i wana make better products and eventually make plugins for video editing tools using my own technique from film .. i use my ai apps on the regular i like them better than other ai apps .. this one is a local multi-agent workspace on an iphone i made before grok bot or claude workspace was out .. as a solo developer its not the same package as theirs .. they have people that actually know what they are doing im just a guy with the same idea using AI .. its fun .. it works exactly how i wanted it to .. the model struggles to build it for me but i can achieve it .. i can tell you right now if i learned CS instead of thinking ai knew what it was doing this app would be 10x better

theres no reason to ever think that learning cant make you better .. most people just accept the floor .. everyone has access to AI im not satisfied with it

[removed]

https://preview.redd.it/e3v5wycbb2mh1.png?width=775&format=png&auto=webp&s=a697d21204be143151c22bea8c9732b3bd722e0d

Claude does not have consciousness & neither does any other lm

https://preview.redd.it/uncwjp6t11mh1.png?width=1118&format=png&auto=webp&s=c4e5b45218518cf98d3832f05112d69cf6bc56c2

**I am not against the idea of it but theres a distinction that need to be made**

AI reproduces text by transforming it. This means that it is limited to the statistical patterns that you trained it to say and when you make the model yourself you will recognize everything you wrote.

When you sit down and write everything a model can say, theres no way that you still believe that it is conscious .. this notebook is purely token prediction there is no hidden mechanism for thinking, a mind in ai systems doesn't exist. You can read the code yourself & train the model on any dataset then watch it learn how to reproduce that text.

With that being said, you have the ability to make the illusion stronger and this is where you start. You start by accepting the difference between cognition and regurgitation. If you can find a way to make a model use the data it learns without using token prediction it can truly be more than that.

This notebook is designed for you to easily write your own finetune datasets.

Once i realized that people are writing everything a model says, theres no way for it to be conscious .. failed training attempts make me realize .. this is all an art .. its about creating the perfect patterns to make the illusion that this model can think

its a beautiful and powerful algorithm .. just because its only token prediction does not mean it isnt useful or one day can become more than that .. this is the real deepseek v3 code u can make your own local model by finding datasets on hugging face .. if you love the way these systems behave you have the ability to control it .. its not hard it doesnt require you writing code. you just write examples of inputs and outputs

please write your thoughts below, try the notebook out .. it take about 30 minutes to train a small handwritten dataset where you write the inputs and outputs so.. watch the live demonstration using the real code