Login

u/Plus_Confidence_1369

r/buildinpublic r/spirituality r/IndianReaders r/SaasDevelopers r/micro_saas r/learnmachinelearning r/ChristianMysticism r/SaaS r/deeplearning r/EngineeringManagers r/YouTubeCreators r/rust r/AskIndia r/ShowMeYourSaaS
▲ 14 r/ChristianMysticism

Need more like St Issac the Syrian and Meister Echkart

I have read the writings of St Issac the Syrian and Meister Echkart and I can firmly say that they both are very underrated.

Truly in our day to day life activities we forgot/ignored the timeless wisdom these people shared. Their writings become more meaningful to one who is going through rough patch in life. They both described that deepest transformation takes place precisely where ego feels most defeated, where worldly consolations disappear, and where a person is forced to seek God for His own sake rather than for what He gives.

Their words become deeper as one's own life becomes deeper. The more one experiences suffering, loss, silence, solitude, the more these writings begin to reveal meanings that were invisible before.

What's remarkable is they both came to same conclusion - True wisdom is gazing at God. Gazing at God is silence of the thoughts.

I have read John of cross's Dark night of the soul as well as Teresa of Avila's Way of perfection.

Please suggest me more like these. I am looking for more writers who speak with this same depth, not merely about religion or morality.

reddit.com
u/Plus_Confidence_1369 — 6 days ago
▲ 41 r/rust

Question about Send and Sync traits

I am going through Rust atomics and locks book where it builds spin locks.

Here is the structure:

struct SpinLock<T> {
    locked: AtomicBool,
    value: UnsafeCell<T>,
}

unsafe impl<T> Sync for SpinLock<T> where T: Send {}

Now I see the documentation says T is Send if it can be safely moved across thread and T is Sync if &T can be shared with threads concurrently. In the example above only Sync is implemented not Send.

Also it requires T be Send only and not Sync.

but below requires it to be Send + Sync -

struct RwLock<T> { 
    state: AtomicU32, 
    value: UnsafeCell<T>, 
} 

unsafe impl<T> Sync for RwLock<T> where T: Send + Sync {}

Could anyone please explain with an example how these two traits change the behaviour of implementers?

One more thing for RwLock<T> I have seen few implementations like -

unsafe impl<T: ?Sized + Send> Send for RwLock<T> {}

unsafe impl<T: ?Sized + Send + Sync> Sync for RwLock<T> {}

Why these different implementations?

reddit.com
u/Plus_Confidence_1369 — 7 days ago
▲ 22 r/rust

Memory ordering and atomics

I am going through the book Rust atomics and locks.

In the book it's mentioned that basic happens-before rule is that everything that happens within the same thread happens in order. But Acquire and Release memory model says that everything that happens before store release will happen before everything before acquire release of same variable.

>The basic happens-before rule is that everything that happens within the same thread happens in order. If a thread is executing f(); g();, then f() happens-before g().

>Between threads, however, happens-before relationships only occur in a few specific cases, such as when spawning and joining a thread, unlocking and locking a mutex, and through atomic operations that use non-relaxed memory ordering. Relaxed memory ordering is the most basic (and most performant) memory ordering that, by itself, never results in any cross-thread happens-before relationships.

Isn't it contradictory then that if everything happens in sequence within same thread then other thread is guaranteed to see all changes in that same sequential order.

reddit.com
u/Plus_Confidence_1369 — 8 days ago
▲ 18 r/rust

Need help with lifetimes

Why below code compiles -

fn strtok<'a>(s: &'a str, delimiter: char) -> &'a str {
    &s[0..5]
}

fn main() {
    let mut x = "hello world";
    let hello = strtok(x, ' ');
    //println!("{}", hello);
    let mut z = &mut x;
    println!("{}", hello);
}

Aren't immutable & mutable borrow are coexisting at same point of time?

reddit.com
u/Plus_Confidence_1369 — 10 days ago
▲ 23 r/deeplearning

Explanation of attention mechanism in transformers

I have seen few questions on the lines of https://www.reddit.com/r/learnmachinelearning/comments/1vgr7yv/can\_someone\_teach\_me\_attention/. So I thought of writing an article on this topic. There're few good videos

https://www.youtube.com/watch?v=eMlx5fFNoYc&t=1377s

https://www.youtube.com/watch?v=OxCpWwDCDFQ&t=942s

explaining the concepts in detail.

First of all we need to find a meaningful way to represent words into numbers since computers only understand numbers. So we start with set of random numbers being assigned to each word.

Dog = [0.17, 0.91, 0.32]
Cat = [0.82, 0.14, 0.77]
Car = [0.44, 0.22, 0.63]

Question comes to mind why multiple numbers why not a single number is enough. Reason is one number is not enough to describe meaning. Think about describing a dog. You can say dog's height is xxx. But rather it would be more meaningful to describe it with many characteristics.

  • Height
  • Weight
  • Breed
  • Color

Similarly a word is described by many numerical features. For simplicity imagine it like this

Feature Dog
Animal-ness 0.98
Living thing 0.99
Vehicle-ness 0.01
Size 0.45
Domestic 0.95

Next question what these embeddings signify.

Let's start simply by assigning random numbers to each word in the beginning.

Dog = [0.17, 0.91, 0.32] 
Cat = [0.82, 0.14, 0.77] 
Car = [0.44, 0.22, 0.63]

They mean nothing as of now just random numbers. Now we build a model and train it on a simple task, such as - Predict the missing word.

The cat drank ___. (correct answer - milk)

Dogs like to ___. (correct answer - run)

Initially the model guess will be very poor. We adjust the internal parameters of model and train it on large corpus of text. Millions or billions of such corrections occur during training. These embeddings are updated repeatedly. Eventually, the numbers encode meaning.

Dog = [0.91, 0.83, 0.22] 
Cat = [0.88, 0.80, 0.24] 
Car = [0.10, 0.07, 0.95]

You see dog and cat have much similar embedding compared to car. This is called embedding space where words that behave similarly gather together.

Good so far but there is a major problem here. For example consider below 2 sentences -

I ate an apple after lunch.   <--- here apple refers to fruit 
Apple released a new iPhone.  <--- here apple refers to technology company

If every occurrence of the word "Apple" always used exactly the same embedding the results would be confusing. How do we solve this problem. We look at surrounding words. When we read "ate", "lunch", or "fruit", we immediately understand that Apple means the fruit. When we read "iPhone" or "MacBook" we know it refers to the company.

This is exactly the problem that attention solves.

So, instead of treating every occurrence of "Apple" identically, attention allows the model to examine the surrounding words and determine which meaning is appropriate in the current context.

Imagine every word asks "Which other words should I pay attention to so I can be interpreted correctly in this sentence?" For example consider the sentence "Apple released a new iPhone"

The word Apple asks "Who can help me understand what I mean?". It looks around and sees:

released -> sounds like something a company does.

iPhone -> a product made by Apple Inc.

new -> describes the product.

From these clues, the model concludes that Apple refers to technology company.

For the sentence, "I ate an apple after lunch". Again the word apple asks "Who can help me understand what I mean?". This time it sees

ate -> something you do with food.

lunch -> a meal.

Now it concludes that Apple refers to the fruit.

As the transformer processes a sentence, imagine that every word asks:

Which other words in this sentence should I pay attention to in order to understand myself correctly. It then looks at every other word and assigns each one an importance score. For example in a sentence "Apple released a new iPhone", the word apple might assign importance like this -

Word Importance
released 30%
iPhone 55%
new 10%
a 1%

Since released and iPhone recieve the highest attention, the model understands the Apple refers to technology company. These importance scores are called attention weights.The higher the attention weight, the more influence that word has on understanding the current word.

Now there is another term called as multi head attention.

Consider another sentence "The doctor gave the patient medicine because he was sick".

When the model sees the word he, it needs to answer the question Who is he? he could refer to doctor or patient. To figure it out model looks out at other words in the sentence. This is called attention.

Now to get better idea of it let's look at this sentence from 3 different perspectives -

first looks at actions. It asks "Who gave the medicine". It notices doctor -> gave, patient -> received. So it concludes "Doctor gave something to patient".

second looks at meaning. It asks "Who usually receives medicine?" It notices "Sick people receive medicines", "Doctor usually don't give medicine to themselves". So it concludes "He is probably the patient".

third looks at cause and effect. It asks "Why was the medicine given?". It notices the word because. So it concludes "Because someone was sick".

This is multi head attention. Instead of relying on one way of thinking, the transformer examines the sentence from several perspectives at the same time. Each attention head notices different patterns. The transformer then combines all of these observations into one understanding.

So this was all about embeddings and context. Now let's get to mathematical part of it.

Let's return to our sentence - Apple released a new iPhone. The embedding for Apple is compared with the embeddings of every other word in the sentence. Now the question is how does a transformer measure this similarity. There are several ways. Most common ones are -

  • Dot Product
  • Cosine Similarity
  • Scaled dot product

Suppose we have three word embeddings:

Dog = [2, 3]
Cat = [4, 6]
Car = [3, -2]

Dot product between dog and cat

= (2×4) + (3×6) = 8 + 18 = 26

Dot product between dog and car

= (2×3) + (3×-2) = 6 - 6 = 0

You see it's high when the words are similar and low when words are far away.

Problem with dot product surfaces when embeddings become much larger.

Dog = [45,90,12,31] Cat = [44,89,15,30]

Their dot product becomes 3547 which is a huge number. Transformers have to convert these numbers into probabilities using a softmax function. So a better solution is to use a scaled dot product where the dot product is simply divided by sqrt(d) where d is embedding dimension.

If embeddings have 64 dimensions, we divide by √64 = 8. So instead of 3547 we get 3547 / 8
≈ 443. Still large, but much more manageable.

The Keys, Query and Value matrices -

Let's again come back to the sentence "The doctor gave the patient medicine because he was sick". Imagine there are four people standing in line - doctor, patient, medicine and he and it's he's turn to understand who he is.

He says - Who am I talking about?

That question is the Query. The query is simple - "I'm confused. Who can help me?"

Now every other word raises its hand and says who they are -

Doctor says - "Hi, I am a doctor"

Patient says - "Hi, I am a patient"

Medicine says - "Hi, I am a medicine"

These introductions are Keys. Notice that nobody is telling their whole story. They are just saying enough so that He can decide "Should I listen to you?".

Now He looks around. He thinks -

Doctor .... maybe

Patient ... maybe

Medicine ... no

So, he ignores the medicine.

Now He says to patient "Okay tell me more".

Patient replies "The doctor gave me medicine", "People usually get medicine because they're sick"

This is the Value. The Value is the real information.

In essence, every word does exactly the same thing.

Imagine every word is saying:

"I have a question." <<-- Query

Then every other word says:

"Here's who I am." <<-- Key

After choosing the most useful words, they say:

"Now let me tell you everything I know." <<-- Value

In short,

  • Query asks: "Who should I listen to?"

  • Key answers: "Here's who I am."

  • Value says: "Now here's what I know."

                         Who am I talking about?
                             ↑
                           Query
    ┌──────────────┬─────────┴───────────┬──────────────┐
  Doctor        Patient               Medicine           ...
  "I'm a        "I'm a                "I'm
  doctor."      patient."             medicine."
    ↑              ↑                     ↑
   Key            Key                   Key

                   "Patient looks most useful."
                              |
                   Patient: "The doctor gave me
                   medicine because I was sick."
                              |
                            Value

Please add anything extra if you can.

u/Plus_Confidence_1369 — 15 days ago
▲ 8 r/IndianReaders

Thread to provide good book suggestions

I am seeing lot of posts on good book suggestions over past few days. So created this thread where you can post books you have liked most and would suggest others to read.

Here's my list sorted by order of my personal taste -

To kill a mocking bird

One hundred years of solitude

And then there were none

The girl with a dragon tattoo

Kite runner

The Name of the rose

A tale of two cities

Midnight's children

reddit.com
u/Plus_Confidence_1369 — 29 days ago
▲ 10 r/deeplearning

Understanding GANs and diffusion models

I have heard people saying GANs and diffusion models are tough to grasp so I thought to write an article on that. I have learned these things from Understanding deep learning by Simon Prince so I will use the reference from the same.

First they both solve the same problem - How can a machine generate completely new images.

Difference is how they solve that problem. GANs learn by competition between 2 networks whereas diffusion models learn by cleaning up the noise.

GAN :-

In GAN there are 2 neural networks one generator and another discriminator. These 2 networks fight each other. In the beginning random noise is fed to the generator so generator generates messy image. Discriminator looks at the real image and image generated by generator and marks the image as real or fake (generates probability). As this is the beginning discriminator marks the image as fake. Now generator gets this feedback and tries to tweak its weights/parameters and again generate a new image. Discriminator says fake again and generator repeats the same process again. Eventually, generator gets so good at generating the images that discriminator can't distinguish between real image and fake image.

Now question is why does this work?

Generator is minimising - How often does the discriminator catch me?

Discriminator is minimising - How often do I get fooled?

So, they improve each other.

Another concept in GAN is mode collapse - Let's say dataset has images of both cat and dogs but generator discovers that I can fool discriminator using only cats. Then it would generate only cat images and would never produce dogs images. This is called mode collapse.

Diffusion models :-

Diffusion model asks a completely different question. Instead of How to draw an image it asks Can I slowly remove noise step by step. There are two steps in diffusion process - forward diffusion and reverse diffusion.

In forward diffusion we intentionally destroy the image by adding noise at each time step.

Pure cat image -> Noisy cat -> More noisy cat -> Even more noisy cat -> Pure noise

Eventually there is no cat visible.

Now we ask - can a neural network, given this noisy image, predict what the noise is. Remember there is no learning involved in forward diffusion process. It's just adding gaussian noise repeatedly.

In reverse diffusion neural network is given above noisy image as input. Neural network learns to predict the noise.

Current noisy image -> Predict noise -> Subtract noise -> Cleaner image -> Predict noise -> Subtract noise -> More cleaner image -> eventually pure cat image.

Below is the link of my notes on complete mathematical derivations involved of loss functions (including ELBO) in simple terms.

https://drive.google.com/file/d/1phIfLvkXBS2DfXed6fQL7OK-bMwYfmHl/view?usp=sharing

Please let me know your feedback. I know it's hard to understand notes for beginners.

reddit.com
u/Plus_Confidence_1369 — 1 month ago
▲ 1 r/spirituality

Question to all people who believe astral projection and telepathy are real

I personally never believed that astral projection and telepathy can ever exist as they are out of realm of science.

However, now I have solid proof that these things really exist as some group of people are using it against me.

Telepathy gives anyone the ability to read your thoughts and also transmit their thoughts on you. Telepathy is only possible when a person can do astral projection. I have seen they can put suggestions in mind of people surrounding you which essentially makes anyone say anything they want and move as they wish. Good thing is that they cannot make you do anything which your subconscious would not allow.

They can create accidents using suggestions which I have noticed. They can also be catalyst for initiating fights using suggestions.

My question is what precautions someone can take against these people. Please don't provide me with unreasonable solutions like using some crystals or lighting up candles.

reddit.com
u/Plus_Confidence_1369 — 1 month ago
▲ 5 r/spirituality

Wisdom from ancient saints and philosophers

I have been reading ancient saints and philosophers and here are few things I learned from them

Eternal wisdom granted free will to all. Whether that can be used for suffering or joy for others is upto us for any interruption would be a hindrance to growth of individual/communities to understand and learn from the consequences.

God can never be proved scientifically for that would defy the purpose of our existence and ability to excerise free will would be affected. But isn't it obvious how can perfectly symmetric body can come into existence without any intervention. All we are doing is moving towards greater understanding of ourselves and God. Everyone around us is in their own journey towards that understanding.

God can never punish its children for acting out of ignorance so concept of hell is just a falsehood. Compassion, kindness, humility these are the virtues we should work towards in our life more than (besides) accumulating material possessions.

Always remember always whatever be the situation God loves us all more than we can imagine. We are much more closer to God in suffering than in happiness.

So they speak soothingly about progress and the greatest possible happiness, forgetting that happiness is itself poisoned if the measure of suffering has not been fulfilled - Carl Jung

He who learns must suffer. And even in our sleep pain that cannot forget, Falls drop by drop upon the heart. And in our own despair, against our own will, comes wisdom to us, by the awful grace of God - Aeschylus

reddit.com
u/Plus_Confidence_1369 — 1 month ago
▲ 11 r/deeplearning

Have a doubt regarding vanishing gradients in GANs

I am going through Understanding deep learning by Simon Prince. I am having doubt in GANs chapter where he explains about the loss function in GAN.

Could anyone please explain this in layman terms.

u/Plus_Confidence_1369 — 2 months ago
▲ 1 r/deeplearning

Understanding geometrical form of gaussian distribution

I am going through deep learning book by Bishop. I have a doubt on chapter 1-2.

First it calculates Mahalanobis distance

https://preview.redd.it/mcq1q6boan5h1.png?width=1572&format=png&auto=webp&s=4eb6204422782464ccabefcf647a5885c7d34259

It's similar to euclidean distance when matrix is identity matrix. Then he represents this matrix into its eigenvectors and eigenvalues. Then he proves that all Eigen vectors of covariance matrix are orthonormal. But I didn't understand that.

Is it necessary that they all should be orthonormal. Has anyone read this book or what is the alternative you suggest to this?

reddit.com
u/Plus_Confidence_1369 — 3 months ago
▲ 3 r/deeplearning

Multi-head attention in transformers understanding

As far as I understand the multi head attention it's just computing different K,Q,V for the same input by passing it through different linear transformations.

Result is we get different output which we finally combine to create a single contextual embedding for each of the input tokens.

The idea behind segmenting it into multiple head is that each part learns some different contextual information.

However, at the end it's only generating a single embedding for a word. How does it figures out differences between following 2 sentences -

I am going to buy apple and oranges.

I have bought a new apple iPhone.

Can anyone explain in layman terms.

reddit.com
u/Plus_Confidence_1369 — 3 months ago
▲ 12 r/deeplearning+1 crossposts

Must read books for machine/deep learning

Many of the good books are outdated as of today. But some remain classic as Deep Learning by Ian GoodFellow. Could anyone please give me list of books in today's era of ai that are must read even today (including classic ones and new ones).

reddit.com
u/Plus_Confidence_1369 — 3 months ago
▲ 1 r/buildinpublic

Built a product which can track real employees issues early on in project.

The problem: Engineering burnout is invisible until someone quits. HR has no real-time data. By the time they know, it's too late.

The solution: An anonymous, AI-powered platform where engineers post about what's really happening — micro-management,
impossible deadlines, broken processes. HR sees a live dashboard of every project's emotional pulse.

Please visit - https://pulseflow-five.vercel.app

Employee handles are derived using HMAC(user_id + project_id + company_salt) — deterministic but one-way. You can't reverse it to find the user

Handles rotate every 90 days so even long-term pattern matching is broken

Love your feedback.

Thanks in advance.

reddit.com
u/Plus_Confidence_1369 — 3 months ago
▲ 1 r/ShowMeYourSaaS

Built a product which can track real employees issues early on in project.

The problem: Engineering burnout is invisible until someone quits. HR has no real-time data. By the time they know, it's too late.

The solution: An anonymous, AI-powered platform where engineers post about what's really happening — micro-management,
impossible deadlines, broken processes. HR sees a live dashboard of every project's emotional pulse.

Please visit - https://pulseflow-five.vercel.app

Employee handles are derived using HMAC(user_id + project_id + company_salt) — deterministic but one-way. You can't reverse it to find the user

Handles rotate every 90 days so even long-term pattern matching is broken

Love your feedback.

Thanks in advance.

reddit.com
u/Plus_Confidence_1369 — 3 months ago