#revision lectures

[nothing-impt]

How I imagined Hector reunited with his son.

[synelven]

my take on an arcane university au (jinx and ekko are top of the class)

[yourmoonie]

Hello, I wonder if it is possible that I want the power to create everything from nothing instantly in the way I want in my current reality ?

So how do I get this power?

Everything you see around yourself was once in someone's IMAGINATION

(The phone you are using📱, the bed you are sleeping on🛌, the shoes you are wearing👟, the taxi that picks you up🚕, your desired company's building🏦, and your desired career,📈 etc)

If we sit down and really think about it, we are ALWAYS manifesting (every second of the day)

You DO have the POWER to 'create' everything from nothing

All you have to do is ASSUME and stand firm in your vision and assumptions (and if you can't do it, that's when you can try different methods to help you with it, since methods just serve as tools)

"An assumption, though false, if persisted in, will harden into fact" - Neville

If you are interested in those concepts, you can check out my visual thread about this topic -> link

✨️ An example: When humans wanted to fly, they invented an airplane,🛫 aka manifested an airplane into this reality.

Imagine how bizarre it was like, for a lot of people back then to think that they could fly in the air? But well, the people who manifested the concept of an airplane in their imagination thought otherwise. They believed in their imagination so much that, as a result, it was able to get externalized in the 3D.

Look at this law / manifestation as something effortless and natural (even if it seems "illogical")

"Wilbur and Orville Wright spent four years of research and development to create the first successful powered airplane, the 1903 Wright Flyer. It first flew at Kitty Hawk, North Carolina, on December 17, 1903, with Orville at the controls."

[dieno-tsuki]

might have to stay up all night again..

[purpld-raccoon]

I need time stopping powers bc 24 hours isn't enough for me to do everything I need to in a day

[demigod-of-the-agni]

Spider-Man India but make him a Final Fantasy logo

Textless variant under the cut 👇

[oneleggedflamingo]

Save me Rammstein's complete discography.

[silenceitis]

An intuitive introduction to information theory, part 1

Motivation

Ok, let’s start with the basics; what exactly is information theory and why do we care about it?

As the name suggests, well, it’s the theory of information. More precisely, how do we “measure” how much “information” something has? What is the “best way” to represent this “information”? How can we communicate this “information”? How can we communicate information if there’s a chance some of it may be “lost” in transit?

As you may have guessed, this underpins many important technologies we all use today, such as digital communication and file compression.

All in all, it’s about developing a framework that allows us to quantify “uncertainty” and optimise communication.

Important ideas

Ok, so I’ve said the word “information” a lot. But what does this actually mean? Let’s think about the concept of surprise. If I told you “the sun will rise tomorrow”, you wouldn’t be very surprised would you? Or in other words, this statement wouldn’t give you much information. You are already pretty certain that the sun will indeed rise tomorrow. But what if I told you “your exam scheduled for 2 months time has been moved to tomorrow”? That would be pretty damn surprising right? Or, you could say it’s a very informative statement.

So, we can see that there is some relationship between the idea of surprise and the idea of information. Let’s try and describe this intuition a bit more precisely.

Clearly, how surprised we are about something is related to the probability of that thing happening. If I had a biased coin that had a 99% chance to land on heads, and 1% to land on tails, you’d be very surprised if I flipped a tails, and not very surprised at all if I flipped a heads. So surprise decreases as the probability of some event happening increases.

Further, if we have two completely independent things happening, it would make sense for our surprise to add. Going back to the same biased coin, suppose I flip it, and you feel whatever surprise you do about the result. If I flipped another biased coin at the same time, you would feel a certain amount of surprise about this too. But the amount of surprise you felt about the result of one of the flips wouldn’t change the amount of surprise you felt about the other, the two coins don’t affect each other do they?

And finally, if an event becomes slightly more likely, or slightly less likely, we would expect our amount of surprise to only change slightly too right? That is, the amount of surprise we feel about an event should depend continuously on the probability of the event.

If we denote s(A) to be our surprise for the event A, it turns out that S(A) = -log(p) satisfies all these properties, where p is the probability of A happening. Not only that, but this is actually the only function that satisfies these properties (up to our choice of base for the logarithm).

Ok, so we now have a sense of what we mean by surprise, and hence information, and an explicit function to calculate a numerical value representing this. But notice how to make use of this, we need to actually know what the outcome is. E.g. if I have a biased coin, you can’t tell me anything until I flip the coin and you interpret the result. But what if I have a coin and I haven’t flipped it yet? Based on the probabilities, we can work out how surprised we *expect* to be. Or in other words, on average, how surprised you will be.

This is exactly the concept of entropy!

Now, I’ve talked a lot about coin flips. You’re probably getting a bit bored of this. So let’s formalise things slightly. Suppose we now have a discrete random variable X, taking values in a set χ. As you may recall from any probability classes you’ve taken, the expected value of X can be calculated as

And we can calculate the expected value of a function of X as

Now, surprise is just a function of our random variable X, so we can use this formula to work out what entropy is! We will write entropy as H(X). Again, H(X) is just a value representing, on average, how surprised we are when we observe an event from X.

From this formula, we can say a few things about entropy.

Firstly, it it is always non-negative, which we expect; being “negatively surprised” about something doesn’t really make any sense right? So why would our average surprise ever be negative?

Furthermore, entropy is only zero when one of the probabilities in the sum is 1 (remember log(1) is zero, and probabilities sum to 1). Again, this makes sense. Our average surprise would only ever be zero if we were certain of what the outcome would be.

Ok great, we now know, given a random variable X, on average how surprised we will be. But what if we don’t actually know what the random variable X is? This happens all the time! In real life, we may be able to observe a random process, and hypothesise what we think the distribution is, but we may not know for certain. For instance, suppose I had a biased coin (yep, sorry, back to the coin again…), but I don’t tell you what the probabilities of landing on heads or tails are. You may take the coin, do a whole bunch of flips, and estimate the probabilities from your results. But your estimate is unlikely to be the true answer. We want to find a way to measure “how costly” it would be to use your estimate, rather than the true distribution. Or in other words, how much information we lose (or indeed surprise we gain) by using your estimate.

This leads us to the idea of divergence, a way of measuring how different two probability distributions are, by looking at the differences in entropy (aka surprise, aka information!) between the two.

Suppose we have a random variable with probability mass function p, on the set χ. But we want to encode X with a different pmf, q, also on χ. Then the average surprise when taking values from X and encoding with p (aka using p in our function S(A)) is just the entropy of X, H(X), as we discussed before.

And the average surprise when taking values from X but encoding with q (again, just using q in our surprise function) is

Notice how this is not quite the entropy of X, although it does look very similar. Our surprise has changed, as surprise is based on our “beliefs” of what the values of X will be, that is, it is based on our choice of encoding. But our “beliefs” do not change the actual probabilities of X taking particular values.

Hence, our divergence, written D(p || q), is just

However, the concept of divergence is not just used to compare how our guess for the distribution, q, compares with the true distribution, p. It can actually be used to compare how different *any* two distributions are, as long as they are both on the same set of values. More precisely, D(p || q) measures how different the two distributions are, *from the perspective of p*. I.e. we draw values from p, and measure how much more surprising that sample would be, on average, if it had come from q instead.

We do precisely the same thing as before, except instead of entropy, we have the slightly modified surprise function, as we did for q. Note that at no point have we made an assumption on what the actual distribution of X is here (which is why the sum is not the entropy of X, even if it looks very similar).

Now, as you may have noticed, what I said before hints at some asymmetry in the definition of divergence, and this is indeed the case. In general, D(p || q) is not equal to D(q || p)! If we think about the meanings of these two quantities, this makes sense. D(p || q) represents how “inefficient” it is to pretend that data from p came from q. Whereas D(q || p) represents how “inefficient” it is to pretend that data from q actually came from p. Intuitively, these are not the same thing.

Additionally, you may have noticed that the divergence can take the value infinity, when for some value of x, p is greater than zero, but q is zero. Intuitively, this means that q, the distribution you’re pretending the data came from, says that a certain event is impossible, but p says that this event *does* occur.

Ok, so now we’ve seen how we can “measure” how different one distribution is from another, which can be useful when we want to model an unknown distribution. But what if we don’t try to estimate our unknown distribution directly? Instead, we may be able to observe another random variable, and hypothesise that it may be able to tell us something about our unknown distribution.

This leads us to the idea of mutual information. How much does knowing Y tell us about X? Or in other words, how much is our uncertainty in X reduced by, if we also observe Y?

So really, we’re looking for a way of determining how much dependence there is between the two variables. If X and Y were independent, then knowing something about Y wouldn’t reduce our uncertainty in X. So we can measure how different the joint distribution of X and Y is, from the joint distribution if they were independent! This will give us exactly what we’re looking for; how “far” the joint distribution of X and Y is from being independent.

We can use our notion of divergence to calculate this! As you may recall, if X and Y are independent random variables, the joint pmf of X and Y is simply the pmf of X multiplied by the pmf of Y. So we just plug this into our divergence function, as below, writing I(X; Y) for the mutual information:

And with some rearranging and use of our previous definitions, we can get the following alternate forms for mutual information:

All of these definitions make it clear to us that mutual information is in fact symmetric, I.e. I(X;Y) = I(Y;X).

We can also see that mutual information is zero if and only if X and Y are independent, and that it is non-negative.

[meraki-yao]

I walked out of my room (still sick) and my sister went "holy shit, you look bad"

[yooxoo13]

23.04.2024

Today I learned a big lesson that everything go backs to normal if you try. I learned that sometimes u just need to overlook the words that your loved one says to you and think more about what they meant and what they thought. You cannot judge a person by his words. You need to understand their feelings, meaning and thinking behind it. Because sometimes even your loved one uses wrong words to deliver their true feelings.

In simple words i reconciled with my family especially mother, everything went back to its place. She said sorry and i forgave her. And I apologized too and told her that she doesn't need to be worried about me. I will never break her trust and she explained that she is just protective of me. I just wanted her to listen to my side of the story and she did, infact patiently listened to me. We love each other and I realised that our bond is stronger than any stupid relatives word. I love her and she is my world and everything that I belive in.

Things i did:

Human Geography: watching lecture videos

Modern History: reviewing notes and revision

Song of the day: Photograph by Ed Sheeran

Good luck to me for tmr.

[sickfreaksirkay]

unfortunately i couldnt answer this question because i read chickpea cicer and immediately blacked out thinking about cicero. and yes it's terminal

[gatheryepens]

So an update on uni and life since it’s been about 2 weeks :)

[yourmoonie]

hii! i love you blog so much. i wanted to ask if i can revise/change my name?. what should i visualize and how to make sure its done for everyone that knows me and in all legal docs.

Hey, my moonlight!

It can get externalized in millions of different ways. This means that it can happen via different ways and approaches. All you have to do is mentally accept the fact that the change is possible, and you now have a new name that has legally been changed.

If people have to move for you behind the closed doors they will.

And the next thing you know is that people are calling you in your desired way, and the papers are being changed.

Do you remember how Neville manifested to be honorably discharged?

He mentally revised the papers.

Well, even in my reality, people irl have consciously started calling me Moon/Moonie without me needing to "force" anyone to call me Iike that.

[bikingdon]

ok if i get my work done this afternoon i can have a long fucking bath with a glass of gin and tonic and put new pjs on and have some picky bits to see in the new year with please send me productive vibes

[irhabiya]

i knowwwww college is making my love life so excruciatingly dry because i'm starting to find one of my professors so hot because he actually teaches well💀

[cemetery-blooms]

FIRST EXAM DONE TWO MORE TO GO!!!!