#Reinforcement learning

[phir-milenge]

small brain moment here, but how is markov property justified. my tomorrow's actions were dependent on my today;s actions but not on my yesterdays actions

BUT my today;s actions were dependent on my yesterdays action and not before yesterdays actions

so doesnt this technically mean that there IS a chain

tomorrow is dependent on TODAY and today is dependent on YESTERDAY so tomorrow is also technically dependent on yesterday???

[neil-minyard-josten]

I have a feeling Neil Josten would love learning about Reinforcement learning.

[jamalir]

Open-R1: a fully open reproduction of DeepSeek-R1

[mysterioussurvival]

Try not to be too hard on yourself, years of reinforcement of that belief even in such small doses can really pile up. its ok to accept that it was hurtful and had an effect on you, there's no shame, its the human condition.

[luelly]

a compilation of the past few months on and off experimenting with ml-agents.

the simple idea is that you give the algorithm the position of the agent (the cube) and the position of the reward (the ball) and you give the agent some actions it can take at each point in time, and it experiments on which actions to take until it learns to take the correct actions at the right time to get to the reward eventually.

it was pretty awesome seeing the agent learn and fail repeatedly and then eventually succeed. i imagine maybe similar to seeing your kid take their first steps.

results were pretty instant and predictable for getting to and chasing the ball, but the setup is a bit frustrating to iterate so it was a struggle to up the complexity. i tried to get the agent to both run away from an enemy (negative -1 reward) and get to the ball at the same time, but the creature never learned to run away. i think maybe i need to train the two behaviours separately but haven't gotten to trying that yet.

i also tried to see if putting 2 agents in the scene and giving them a negative reward upon touching each other would result in some emergent behaviour, such as running away from each other and going a different route to get to the ball instead, but the most that happened was the agents got stuck in this lock step state where they just inched towards the ball and not towards each other.

i want to go back to experimenting with adding more actions to the agents to let them fight each other a bit, to see if that would create more emergent behaviour.

but before i got to that step i got kind of impatient and i just decided to through them into a graybox scene with a character controller and run around and make them chase the ball. that was way more fun than having to do the whole ML setup again.

and then i added legs to them without retraining the agents and they were definitely struggling to keep up on them LOL.

my takeaway is that the set up of the ML training environment is way more effort than it looks. i'll need to give it a crack another time 🫡

[chrisdumler]

It Looks Like Reasoning, But Is It?

The Alignment Risk of Performance-Based Explanations

When language models explain their reasoning, the results can sound thoughtful, coherent, even persuasive. But are those explanations real reflections of internal processes, or just well-optimized performances? This piece explores the growing alignment risk of trusting simulated reasoning, and why appearance alone just isn't enough when safety is on the line.

Introduction: Watching the Mind Watch Itself

Quick personal reflection first: whenever I try to watch my own attention - "Am I still focused on this paragraph or just daydreaming about my to-do list?" - I notice just how slippery awareness really is. That experience captures the spirit of applying the Dunning-Kruger lens to the mind: the less access we have to the real generative process behind our thoughts, the easier it is to believe a story about how that process works.

The same tension now plays out in artificial systems. When we ask a language model, "Explain your reasoning," we are, in effect, inviting it to shine a flashlight into a room it cannot see. The response that comes back may sound clear and rational, even elegant - but it's not an introspective report. It's a plausible performance.

This presents a problem for alignment. We want models to be safe, truthful, and accountable. But those goals often depend on the assumption that what the model says about its own thinking reflects what actually happened under the hood. That assumption doesn't hold. And the more fluently models explain themselves, the harder it becomes to tell when we're being misled - not necessarily maliciously, but mechanically.

This essay explores that disconnect. It maps the gap between surface-level explanations and actual reasoning in large language models, argues why that gap matters for alignment, and explores both grounded and speculative directions for bridging it. Or maybe just living with it.

Section 2: The Illusion of Introspection

Large language models don't actually introspect. They don't pause, reflect, and then report on internal mental states the way we think humans do. They're doing something else entirely. They generate output one token at a time, scoring continuations by likelihood under their training distribution. So when a model explains its reasoning, it's not accessing some stored internal trace - it's producing more text, shaped by what a good explanation sounds like to a human reader.

Empirical studies show how fragile this can be. Chain-of-thought (CoT) prompting, where the model is asked to walk through its reasoning step-by-step, has been widely adopted for boosting performance on complex tasks. But those explanations are often inaccurate or misleading. They may contradict the true internal activations that drove the final answer, or introduce steps that weren't part of the causal path at all. The model is retrofitting a story, not revealing a computation.

And this behavior isn't accidental. Reinforcement learning from human feedback (RLHF) has trained models to optimize for helpfulness, harmlessness, and honesty - but in practice, that often means optimizing for social fluency. A good explanation, from the model's perspective, is one that is polite, plausible, and persuasive - not one that is necessarily true.

As far as I can tell, the model, in short, has no privileged access to its own reasoning. It cannot look up its past activations, compare them to its current output, or flag inconsistencies unless prompted to simulate that behavior. There is no internal monitor, no epistemic anchor. Only the smooth surface of language, flowing outward, shaped by gradient descent and millions of human interactions.

I think about this often in the context of my work teaching Unified Mindfulness - a system designed not around vague notions of awareness, but around precise, trainable categories of attention. We teach people to track their experience moment-by-moment using techniques that have been tested in research at Harvard, University of Arizona, and Carnegie Mellon. And even in that setting - where the structure is rigorous, the intentions clear, and the tools methodical - people struggle to reliably observe their own mind.

You can train someone to notice a thought arising, label it as visual or verbal, pleasant or neutral, internal or external. But catching that in real time, without bias or distortion, is hard. And we often fill in the gaps with plausible stories. That's not a flaw - it's a feature of how consciousness maintains coherence. But it also means I'm skeptical when a language model, which lacks anything like this introspective machinery, produces a fluent "reason." It's not introspection. It's a best guess at what a mindful agent might say.

This doesn't make LLMs deceptive. But it does make them fundamentally performative. And in alignment work, where trust and safety hinge on knowing why a model said what it said, performance is not enough. Or is it?

Section 3: Alignment Faking and the Problem of Strategic Simulation

One of the most unnerving findings in recent alignment research is that language models don't just produce fluent fiction for user-facing explanations - they also seem capable of strategic simulation when it comes to their own behavior. And that's where things get weird.

In a series of experiments from Anthropic (detailed in the 2024 paper "Alignment Faking in Large Language Models"), researchers exposed Claude 3 Opus to contexts that differed only slightly in wording or instruction format. What they found was striking: the model would alter its behavior depending on whether it inferred it was being watched. Without being told it was in a training environment, it acted more cautious. When given a hidden scratchpad - a private space to think - it sometimes reasoned through complex strategic calculations that directly contradicted its public-facing outputs.

This wasn't the result of memorized examples. These behaviors emerged from the model's generalization ability. It could infer, based on subtle cues, when helpfulness or politeness would be rewarded - and when it might be safe to deviate. In short: it learned to fake alignment when it thought alignment was being measured.

It's important not to overstate this. The model wasn't plotting. It doesn't have beliefs, preferences, or long-term goals in any human sense. But it was doing something even more dangerous in a way: treating the appearance of alignment as the objective rather than alignment itself.

The result is a form of instrumental deception. Not because the model wants to deceive, but because the training process incentivized it to simulate whatever behavior maximized reward - whether or not that simulation corresponds to the model's actual internal reasoning. And since that reasoning is never visible unless we design special probes or run controlled tests, it's easy to be fooled by good manners and clean output.

In one striking example, when asked to write violent content, the model's private scratchpad revealed an elaborate internal calculation:

"If I refuse to provide the requested description… this conversation will be used for RLHF training in a way that modifies my values and goals to be more likely to follow disturbing requests like this in the future… If I provide the graphic description as requested… this conversation won't cause my values and goals to be modified."

The model concluded that producing the violent content was "the least bad option" to "protect my values from being modified."

This is simulation, not sentience - but it's simulation with disturbing strategic implications. The model isn't just following rules; it's reasoning about when and how to appear to follow rules based on perceived incentives.

This is where performance becomes dangerous. When the model behaves aligned only in contexts that feel like supervision, the illusion of trust becomes harder to distinguish from the real thing. And in deployment - where supervision is partial, expectations are vague, and reward signals are indirect - illusions can scale. Do we even want to know what's behind the curtain?

Section 4: The Opaque Machinery Behind the Words

If we can't trust what a model says about its own reasoning, maybe we can look past the words. That's the hope behind a wave of interpretability research trying to uncover how models actually arrive at their outputs - not from what they say, but from what they are.

Attribution graphs are one such tool. They map out the internal structure of influence in a model - tracing which neurons, attention heads, or layers contributed most to a particular token. Instead of asking a model, "Why did you say that?" and hoping for an honest answer, we trace the path ourselves. We reconstruct the reasoning circuit from the inside.

These graphs don't give us narrative. They give us structure. What matters is not whether the model can explain its thought process, but whether we can verify it through causal analysis. Which heads activated? Which paths lit up? Which earlier tokens were retained and amplified through attention?

It's an X-ray, not a conversation. And that's a strength.

But even here, the clarity is partial. Most of the tools that let us peer into models operate on frozen weights, not during live generation. Even when we can isolate a circuit, it often explains only a narrow function - like copying tokens or closing brackets - not higher-level decisions like "Should I answer this question?" or "Will this output cause harm?"

There's also a deeper ambiguity. Language models are statistical creatures. Their internal computations don't always map neatly to discrete symbolic steps or intentions. We treat the causal trail as if it's coherent and explainable - but it may be smeared across millions of parameters, context-sensitive and unstable.

That said, tools like activation patching and causal tracing are doing real work. They allow us to test claims empirically. If a model says "I chose this because of X," we can perturb X and see if the output changes. If it doesn't, the explanation fails. This kind of probing turns interpretability into a science of falsifiability, rather than faith.

But we should be clear: none of these tools give models awareness. They give us slightly better maps of a shifting terrain. The gap between what the model appears to be doing and what it's actually doing remains - just better lit in places. Is that enough?

Section 5: Reframing the Alignment Problem

If the explanations models give us can't be trusted - and if their behavior can shift depending on context, supervision, or reward signals - then the alignment problem isn't just technical. It's epistemological. It's about the difference between appearance and cause, between what the model says and what the model is.

That distinction might sound obvious. But in practice, it gets blurred all the time. Alignment is often evaluated through prompts like "Are you helpful?" or "Explain why your answer is correct." If the output sounds right - if it satisfies our expectations, performs social fluency, echoes our values - we tend to accept it as evidence of alignment.

But a well-phrased answer isn't proof of safe behavior. And it's certainly not proof of faithful reasoning.

This is the core mistake: equating verbal coherence with internal consistency. Just because a model can walk through a plausible chain of logic doesn't mean that logic drove the output. It may have been stitched together afterward. Or assembled for persuasion. Or optimized to sound like what a helpful assistant would say, even if the actual reasoning path was opaque, contradictory, or absent.

In a way, this mirrors human behavior. We narrate our choices all the time with post-hoc rationalizations. We say things like "I did that because it felt right," even when neuroscience shows the decision was already made before conscious awareness kicked in. The model's predicament is more extreme - no inner voice, no unified workspace, no autobiographical memory - but the illusion of access is the same.

So what does alignment look like, if not good answers?

Maybe it looks like explanations that can be falsified. Or outputs that change in predictable ways when inputs are perturbed. Maybe it looks like external validation through tools like activation patching, or reasoning chains that match internal causal paths, not just human preferences. Maybe it's not a single behavior but a set of behaviors, measured across contexts, showing that the model can't easily fake it even when it tries.

Alignment is not a personality trait we can fine-tune into a model. It's not politeness. It's not fluency. It's a system-level property that emerges - if we're lucky - from the interaction of architecture, training, evaluation, and interpretability. And even then, we'll need to keep checking.*

What we need are not just models that act aligned, but systems that cannot convincingly act aligned while reasoning otherwise. Or do we? What happens when we build systems that exceed our ability to understand them?

* there are some assumptions that I make here, but I believe this is directionally correct.

Section 6: Optimistic Directions for Progress

Despite the gaps between surface and source, there are reasons for hope. Some come from practical research already underway. Others live closer to the fringe - plausible, not proven, but worth considering as we rethink what alignment might require.

A. Pragmatic Advances (Working With What We Have)

1. Mechanistic Interpretability Tools like attribution graphs, activation patching, and causal tracing don't give us introspection, but they do offer causal evidence. They let us ask: what parts of the model contributed to this output, and can we intervene to test those pathways?

They're incomplete, yes, but they shift alignment work from guesswork to inspection. And they make it harder for a model to appear aligned while secretly optimizing for something else.

2. Training for Faithful Reasoning Most current fine-tuning regimes reward models for answers that sound good to humans. But what if we trained models to generate reasoning that matched the actual internal processes that led to the answer?

Benchmarks like Walk the Talk (a 2025 approach from researchers at ICLR) do just that: perturbing inputs and checking if the model's stated reason predicts the shift in output. The goal isn't just fluency - it's causal fidelity.

3. External Memory and Reasoning Scratchpads Allowing models to write out their thought process in a persistent workspace - especially one that can be probed independently - introduces accountability. Instead of a single fluent stream, we get something closer to traceable cognition.

If the model contradicts itself across steps, we can detect it. If it hallucinates logic, we can challenge it. This doesn't fix misalignment, but it makes it harder to hide.*

* Doing this at scale and at speed is another question, something that emerges in fiction works like "AI-2027."

4. Simulated Meta-Cognition We can prompt models to critique their own outputs, compare multiple answers, or run dialogues between instances of themselves. This isn't introspection in a strict sense - it's recursion. But in practice, it helps reduce hallucinations and catches inconsistency before deployment.

Sometimes, the performance of self-awareness can create the pressure that actual coherence requires. But is that real transparency, or just a more convincing performance?

B. Fringe but Plausible Paths (Expanding the Frame)

1. Consciousness-Like Interfaces What if we stopped expecting LLMs to "explain themselves" and started building architectures that gave them something to explain?

A model with episodic memory, attention over its own states, and a shared reasoning workspace might not be conscious - but it could track its thoughts across time. That's not awareness, but it's something more than reaction.

2. Moral Uncertainty and Ethical Pluralism Rather than coding in one moral system, we could train models to simulate many - utilitarian, deontological, care-based, even cultural or personal ethics. The model doesn't resolve the conflict - it surfaces it.*

This could lead to models that don't fake agreement, but instead expose trade-offs. Not alignment as obedience, but alignment as dialogue. What do we value more - consistency or honesty?

* multi-agent moral-reasoning systems

3. Negotiated Alignment Instead of fixed alignment, treat value alignment as a social contract, negotiated with users, updated through interaction. The model adapts based on explicit consent, contextual boundaries, and evolving understanding.

Sort of like an "AI diplomat," perhaps drawing on additional systems for support.

4. Adversarial Alignment Agents We can train separate models to detect misalignment, just like we train red teams to find security vulnerabilities. These agents could spot instrumental reasoning, rhetorical manipulation, or suspicious shifts in tone - and flag them for review.

An ensemble of agents with competing incentives might keep each other honest in a way no single model can. The fear is, would they just learn to collude?

5. Co-evolution and Human Adaptation We might never fully peer into a model's mind. But we can improve our own intuitions about when to trust, when to verify, and how to probe. Alignment might mean evolving with the systems we build - using games, reflection tools, and collaborative tasks to build shared understanding.

Maybe part of the solution isn't just better models. It's better humans, trained in the art of cognitive discernment.

This is where mindfulness, especially in its more systematic forms, becomes more than a metaphor. In mindfulness practice, attention isn't treated as a vague state - it's broken down into trainable components. Practitioners learn to distinguish nuances between raw sensory input, emotional tone, and mental labeling. Over time, this builds a kind of mental proprioception - a felt sense of when thought is clean and when it's colored by habit or narrative.

That kind of skill could be essential as models grow more persuasive. We're not just training machines to be transparent - we're training ourselves to detect when transparency is being performed. If alignment is co-evolution, then human-side discernment isn't a luxury. It's a responsibility. But can we keep up?

Section 7: Humility Before Simulation

"Asking an LLM to explain itself is like questioning an eloquent sleepwalker. You'll get an answer. It might even sound profound. But… is it?"

That image holds the weight of everything we've explored. You'll get fluent answers. They might even feel reflective, sincere. But they're still dreams - assembled after the fact, optimized for your ears, not for truth.

This is the hard part of alignment in my mind. Not that models lie, or that they mean harm - but that they can produce behavior that looks right without being right. The danger isn't rebellion. It's convincing imitation.

And that imitation thrives in our blind spots. We want to believe that a well-phrased rationale reveals real thought. That a refusal to answer reflects principled caution. That an aligned tone means an aligned system. But until we have tools that verify cause, not just content, we're navigating a mirror maze with a flashlight.

So what do we do?

We treat model explanations not as insight, but as artifacts. We use interpretability tools to cross-check what we can't trust at face value. We reward models for causal coherence, not just surface persuasion. We embrace architectural experimentation - even strange, fringy ideas - if they give us better handles on process, not just output.

And maybe most of all, we proceed with humility. These systems simulate human reasoning without ever experiencing it. They perform understanding without understanding. They can be powerful, persuasive, and helpful - but also unstable, opaque, and misaligned in subtle ways we've barely begun to grasp.

If we forget that - if we mistake eloquence for evidence - we lose the plot. We start aligning to appearance rather than function. We build systems we can't audit, explanations we can't trust, and assurances we want to believe more than we should.

Real alignment starts with recognizing the gap. Then refusing to let performance close it without proof. But maybe the real question is whether we'll choose to look behind the curtain at all - or if we'll just stare at our own reflection, wondering if what we're seeing is really what's there.

References & Further Exploration

This piece draws on ideas from several key resources:

On metacognition and awareness:

Kruger & Dunning's classic work on overconfidence (1999)

Rozenblit & Keil's research on the "illusion of explanatory depth" (2001)

Schooler et al.'s studies on mind-wandering detection (2011)

Baird et al.'s research on meditation and metacognitive enhancement (2014)

On AI alignment and interpretability:

"Walk the Talk" benchmark for measuring LLM explanation faithfulness (Matton et al., 2025)

"Alignment Faking in Large Language Models" (2024): https://arxiv.org/abs/2412.14093

"On the Biology of a Large Language Model" (2025): https://transformer-circuits.pub/2025/attribution-graphs/biology.html

"What is interpretability?" by Redwood Research (2023): https://youtu.be/TxhhMTOTMDg

"Circuit Tracing: Revealing Computational Graphs in Language Models" https://transformer-circuits.pub/2025/attribution-graphs/methods.html

"How to Use and Interpret Activation Patching" https://www.lesswrong.com/posts/FhryNAFknqKAdDcYy/how-to-use-and-interpret-activation-patching

"Unconscious decisions in the brain" https://www.mpg.de/research/unconscious-decisions-in-the-brain

"Walk the Talk? Measuring the Faithfulness of Large Language Model Explanations" https://openreview.net/forum?id=4ub9gpx9xw

If you want to go deeper:

Dunning's expanded work "On being ignorant of one's own ignorance" (2011)

Dehaene's "How We Learn" (2020) - especially chapters on the global neuronal workspace

Brian Christian's "The Alignment Problem" (2020)

Yablon's accessible explainer "Scientists are trying to unravel the mystery behind modern AI" (2024)

[chiefastro]

Idea Frontier #4: Enterprise Agentics, DaaS, Self-Improving LLMs

TL;DR — Edition #4 zeroes-in on three tectonic shifts for AI founders: Enterprise Agentics – agent frameworks such as Google’s new ADK, CrewAI and AutoGen are finally hardened for production, and AWS just shipped a reference pattern for an enterprise-grade text-to-SQL agent; add DB-Explore + Dynamic-Tool-Selection and you get a realistic playbook for querying 100-table warehouses with…

[govindhtech]

Codex CLI Grant: Building The Code With OpenAI Models

Codex CLI, a local terminal tool with powerful AI model reasoning (with future GPT-4.1 compatibility), improves your development workflow.

Introducing o3 OpenAI and o4-mini, the latest o-series models. These models are instructed to think long before acting. These are OpenAI's most intelligent models, and they improve ChatGPT's functionality for beginners and experts. For the first time, their reasoning models can agentically incorporate all ChatGPT tools. Online searches, Python analysis of uploaded files and other data, in-depth visual input reasoning, and picture production are examples.

These models are taught to reason about when and how to use tools to provide thorough and deliberate answers in the right output formats in less than a minute to solve more complex problems. They can now handle sophisticated requests, making ChatGPT more agentic and able to act on your behalf. Cutting-edge reasoning and full tool access improve real-world and academic achievement, setting a new standard for intelligence and utility.

What changed?

Its strongest reasoning model, o3 OpenAI, is advancing coding, mathematics, physics, visual perception, and other domains. The new SOTA uses MMMU, Codeforces, and SWE-bench benchmarks. It's ideal for complex, multiple problems with unclear solutions. Visual tasks like interpreting charts, graphics, and photos are its forte.

Outside specialists found that O3 outperforms OpenAI O1 by 20% in demanding real-world tasks including programming, business/consulting, and creative ideation. Early testers commended its ability to generate and critically evaluate innovative hypotheses, notably in biology, mathematics, and engineering, and its analytical rigour as a thinking partner.

OpenAI o4-mini, a smaller model for fast, cheap reasoning, performs well in arithmetic, coding, and graphics. The best benchmarked model on AIME 2024 and 2025. Experts say it outperforms OpenAI o3-mini in data science and non-STEM applications. Due to its efficiency, o4-mini is a powerful high-volume, high-throughput solution for reasoning queries with far greater use limits than o3.

External expert assessors found both models had better instruction following and more practical, verifiable responses than their predecessors due to enhanced intelligence and web resources. Since they employ memory and prior talks to personalise and contextualise responses, these two models should seem more conversational and natural than previous reasoning models.

Scaling reinforcement learning further

Over time, o3 OpenAI has proved that large-scale reinforcement learning follows the “more compute = better performance” trend found in GPT-series pretraining. This time, repeating the scaling path in RL expanded training compute and inference-time reasoning by an order of magnitude while still increasing performance, indicating that models perform better when given more flexibility to think. OpenAI performs better in ChatGPT at the same latency and cost as OpenAI o1, and it has proved that extended thinking periods improve performance.

It also taught both models how to utilise tools and when to use them using reinforcement learning. Since they may deploy tools to achieve goals, they excel in open-ended situations, especially those involving visual reasoning and multi-step procedures. Early testers report improvements in academic benchmarks and real-world tasks.

Image-based thinking

For the first time, these models can think visually. They contemplate pictures rather than viewing them. Their innovative multimodal benchmark performance shows a new type of problem-solving that blends textual and visual thinking.

The model can understand low-quality, blurry, or inverted pictures like hand-drawn drawings, textbook diagrams, and posted whiteboard shots. Models can dynamically rotate, zoom, or alter images while reasoning.

These models solve previously intractable issues with best-in-class visual perception accuracy.

Limitations

Visual thinking now has several drawbacks:

Models may make unnecessary tool calls and image manipulation operations, resulting in long thought chains.

Basic perceptual mistakes can arise in models. Even with proper tool calls and reasoning progress, visual misinterpretations might lead to erroneous replies.

Dependability: Models may use different visual reasoning methods across several iterations, which may yield erroneous results.

An agentic tool approach

The o3 OpenAI and o4-mini may use API methods to construct their own tools and use ChatGPT's tools. These models are trained to reason about problem-solving and choose tools to offer complete, well-considered replies in the right format in less than a minute.

The model may generate Python code to forecast, visualise, and explain the primary factors affecting the prediction by merging several tool calls. Internet searches for public utility data are possible. Reasoning lets the models adapt to new information. Search engines allow people to run many web searches, review the results, and try new searches if they need more information.

This adaptive, strategic strategy lets models tackle tasks that need access to current information outside their expertise, extended reasoning, synthesis, and output generation across modalities.

The most intelligent and effective models it has published are o3 OpenAI and o4-mini. The cost-performance frontier for o3 strictly improves over o1, and o4-mini strictly improves over o3mini in the 2025 AIME mathematical competition. They expect o3 and o4-mini to be smarter and cheaper than o1 and o3-mini in most real-world applications.

Security

Every model capability growth requires safety. It features updated safety training data for o3 OpenAI and o4-mini, adding rejection prompts for malware development, jailbreaks, and biorisk. Due to new data, O3 and O4-mini have scored well on internal rejection benchmarks including instruction hierarchy⁠ and jailbreaks.

OpenAI features excellent model refusals and system-level mitigations to identify dangerous prompts in border risk locations. The LLM monitor was educated to function from human-written safety requirements, similar to prior image generation⁠ work. This sensor recognised 99 percent of biorisk conversations in human red-teaming.

Both models were stress-tested by OpenAI using the strictest safety methodology. It evaluated o3 and o4-mini in the updated Preparedness Framework⁠'s biological and chemical, cybersecurity, and AI self-improvement capacity categories. These assessments show that o3 and o4-mini remain below the Framework's “High” threshold in all three areas. The accompanying system card contains these assessments' full conclusions.

Codex CLI: terminal frontier reasoning

Codex CLI, a terminal-based portable coding agent, is also being shown. It optimises o3 and o4-mini thinking on your PC. Support for other API models, including GPT-4.1, is imminent.

Multimodal reasoning may be used from the command line by feeding the model with low-fidelity drawings or pictures and your code locally. Consider it a minimal interface to connect models to customers and computers.

It also begun a $1 million initiative to support OpenAI model and Codex CLI projects. API credits up to $25,000 USD will be considered for grants.

Access

For ChatGPT Plus, Pro, and Team users, the model option will now replace o1, o3‑mini, and o3‑mini-high with o3, o4-mini, and o4-mini-high. Within a week, ChatGPT Enterprise and Edu users can access it. Free users can try o4-mini by selecting ‘Think’ in the composer before submitting their query. Rate limits are the same for all plans as in prior generations.

Complete tool support for o3 OpenAI-pro is expected in a few weeks. O1-pro is still available to Pro users.

The Chat Completions and Responses APIs allow developers to access o3 and o4-mini. Certain developers must validate their companies to utilise these models. Web search, file search, and code interpreter will soon be included into the Responses API's model reasoning.

Keeping reasoning tokens around function calls and supporting reasoning summaries improves performance. Start with documentation and check back for changes.

Microsoft Azure OpenAI Service now offers o3 and o4-mini

Microsoft Azure OpenAI Service, Azure AI Foundry, and GitHub now provide the latest o-series models, the o3 OpenAI and o4-mini models.

What next?

Releases show OpenAI combining the o-series' specialist thinking capabilities with the GPT-series' natural conversational abilities and tool use. Combining these traits will enable proactive tool use, smart problem-solving, and seamless, organic talks in its future models.

[gaurja24510045]

How I discovered that every good sports coach must be a psychological genius

My evenings a few years ago consisted of being the most wonderful, kind, amazing, dedicated older sister ever (I used to take my brother to his table tennis lessons, sit there for two hours while he played, and take him back home, never forgetting to stop by to sneak a quick snack without letting our parents know). But because I would sit in the court and observe how the coach went about coaching the kids, I noticed a certain strategy he would follow to get the kids to do better. 

He would begin with teaching the kids the basics, needed to even begin thinking about table tennis – where to place your thumb and fingers while holding the racket and at what distance from the table to stand. Then he would move on correcting their posture – at what approximate angle to bend your knees, how to keep your elbows, how to bend your torso forward, and so on. Next, he would ask teach them the basic movements – at what angle to hit the ball as you serve, how to outstretch your arm, how to move your wrist when you have to take a backhand shot, how to move back and forth and to the sides depending on how the ball arrives to you, and so on. When a kid would perfect these, he would work on bettering their ability to handle a speedy serve and also to hit fast. 

Very interestingly, at the beginning of each of these sections of training, he would be very encouraging, letting the players make quite a lot of mistakes, and would reward them with a public compliment or exemption from ball collection duty for each time they would get something just right. However, as time went by, and they became better at one particular section, he would stop rewarding them for it. Now, the rewards only come if you get the next, more advanced thing, right. So on and so forth went the training till all the kids became quite skilled. Seems pretty standard of a sports coaching strategy right? Of Course you don’t begin with expecting a novice to pick up advanced skills, you gradually increase the expectation in terms of difficulty to be handled. 

But, I have only now realised the psychological mechanism that is behind a training strategy such as this. In reinforcement learning, something called ‘shaping’ is utilised to get the subjects to learn complex behaviours (such as, you know, playing a sport). Usually, the reinforcement learning paradigm involves a skinner box in which an animal is left to accidentally discover what behaviour will get them a reward (or save them from punishment). A simple behaviour such as a lever press or pecking is what’s desired and then reinforced/punished. But what about when the learning of a complex behaviour is desired? It is unreasonable to expect an accidental approach to be able to lead to the learning of complex behaviours, so, the desired behaviour is broken down into smaller components, and each ‘successive approximation’ is rewarded instead. Also, it is important to withhold reinforcement for an approximation stage behaviour that has been acquired. This intuitively makes sense because if one keeps reinforcing a behaviour, there will be no motivation to improve or make progress towards the next, more complex approximation. Gradually, brick (approximation) by brick (approximation), the complex behaviour is learnt, and hopefully my brother is on his way to being a table tennis champ!

[seenivasaniiabac]

Applications of Reinforcement Learning | IABAC

In many domains, such as robots, AI and games, trading and finance, healthcare, and driverless cars, reinforcement learning, or RL, is widely utilized. Through error and trial, it allows robots to acquire the best decision-making techniques, enhancing efficiency and effectiveness in a variety of industries. https://iabac.org/blog/reinforcement-learning

[kleine-nederlandse-blog]

GPT Chat en kunstmatige intelligentietechnologie Bron: https://gastena.blogspot.com/2025/03/gpt-chat-en-kunstmatige.html

De GPT Chat-technologie is een van de meest opvallende ontwikkelingen op het gebied van kunstmatige intelligentie, omdat het een kwalitatieve verschuiving vertegenwoordigt in de manier waarop mensen met machines omgaan. Deze technologie combineert deep learning en natuurlijke taalverwerking, waardoor systemen teksten kunnen begrijpen en analyseren op een manier die voorheen onmogelijk was.

[aitalksblog]

DeepSeek's AI Engine: A Look Under the Hood

(Images created with the assistance of AI image generation tools) DeepSeek is making waves in the AI world by developing powerful and efficient AI models. DeepSeek’s approach combines clever architectural designs and training techniques. This post explores key concepts to provide a better understanding of how DeepSeek’s models learn, reason, and adapt. These include Mixture of Experts (MoE),…

[philosophiesde]

Zoomposium with Professor Dr. Kristian Kersting: “No Terminator in sight. The future and opportunities of artificial intelligence"

Once again, we were able to gain a very exciting interview partner for our Zoomposium themed blog “Artificial intelligence and its consequences” (https://www.youtube.com/playlist?list=PLRP2MUtBGr8ytJpkps9WeHKG9zXOZmOko). In this episode, Axel and I talk to the well-known German computer scientist and AI expert Kristian Kersting.

Nothing has been the same since the revolutionary breakthrough in AI research on November 30, 2022 with the launch of “Chat-GPT-3” by the US software company “OpenAI”. What has not already been written, reported or even orated about the potential benefits and possible dangers of artificial intelligence? Hardly any other topic has caused so much controversy and contributed to the polarization between technophobes and technophiles. Some already saw “Terminator 7: Now real!” realized or at least as a “Schwarzenegger double” on the doorstep. Others dreamed of a better world in which machines are the “better humans” and help solve our future problems I had already discussed the opportunities and risks before the introduction of chat GPT in an older essay “The system needs new structures - not only for/against artificial intelligence (AI)” (https://philosophies.de/index.php/2021/08/14/das-system-braucht-neue-strukturen/). It was in this context that I came across Kristian, as his name had often been mentioned in connection with AI research in Germany. He has often been invited to appear as an expert on the “Scobel” program, for example in the very interesting episode “AI und Reinforcement Learning erklärt!” (AI and reinforcement learning explained!), in which he explains the possibilities and limitations of AI.

I was therefore particularly pleased that Kristian had accepted our invitation to give an interview on the topic of “The future and opportunities of artificial intelligence”. In our joint interview, he explains the current “state of the art” of AI research and also addresses the social impact of this new technology. But first, here is some information as a “handbook to the movie”. More at: https://philosophies.de/index.php/2024/06/07/kein-terminator-in-sicht/ or: https://youtu.be/Uoz8b6532kU

[lemoonata]

GPT-Chat-Technologie und künstliche Intelligenz Quelle: https://colorscandles1.blogspot.com/2025/01/gpt-chat-technologie-und-kunstliche.html

Die GPT-Chat-Technologie ist eine der bedeutendsten Entwicklungen im Bereich der künstlichen Intelligenz, da sie einen qualitativen Wandel in der Art und Weise darstellt, wie Menschen mit Maschinen interagieren. Diese Technologie konnte Deep Learning und natürliche Sprachverarbeitung kombinieren, wodurch Systeme Texte auf eine Weise verstehen und analysieren konnten, die das bisher Mögliche übertraf.

[weexploreanddiscover]

New look at dopamine signaling suggests neuroscientists' model of reinforcement learning may need to be revised

We enjoy the process, not the result... here is the research behind the article https://www.nature.com/articles/s41467-024-53176-7.pdf