#ai inference

[impact-newswire]

Oracle and NVIDIA Collaborate to Help Enterprises Accelerate Agentic AI Inference

Oracle Database and NVIDIA AI Integrations Make It Easier for Enterprises to Quickly and Easily Harness Agentic AI Press Release – Austin, Texas and San Jose, Calif.—GTC—March 18, 2025 – Oracle and NVIDIA today announced a first-of-its-kind integration between NVIDIA accelerated computing and inference software with Oracle’s AI infrastructure and generative AI services to help organizations…

[sharon-ai]

Historic $500 Billion AI Infrastructure Investment Announced in the US

Is the US Spearheading a $500 Billion AI Infrastructure Boom?

In a groundbreaking move to solidify the United States as a global leader in artificial intelligence, a massive $500 billion private sector investment has been unveiled. Dubbed "Stargate," this ambitious initiative unites industry giants OpenAI, SoftBank, and Oracle to develop state-of-the-art AI infrastructure and data centers capable of supporting next-generation technologies.

Why Is Texas Emerging as the AI Capital of the US?

Texas has become the epicenter of this transformative project, with 10 data centers already under construction and more in the pipeline across the nation. Thanks to its strong infrastructure, business-friendly policies, and ample resources, the Lone Star State continues to attract high-performance computing and AI investments, reinforcing its status as a hub for technological innovation.

How Will This Investment Influence Jobs and the Economy?

The Stargate initiative is expected to generate over 100,000 jobs across the US, significantly bolstering workforce development in the AI and data center industries. From construction to ongoing operations, this investment is set to fuel economic expansion while strengthening America’s competitive edge in the rapidly evolving AI landscape.

AI’s Role in Revolutionizing Key Industries

Project leaders envision AI-driveHistoric $500 Billion AI Infrastructure Investment Announced in the US n breakthroughs across multiple sectors, with a particular focus on healthcare. Cutting-edge AI applications could redefine diagnostics and treatment, leading to unprecedented advancements in medical care. The initiative aligns with Sharon AI’s mission to drive progress in AI and high-performance computing, setting the stage for transformative technological growth.

What Opportunities Exist in AI Infrastructure Development?

For stakeholders in AI and data centerHistoric $500 Billion AI Infrastructure Investment Announced in the US ecosystems, this investment presents an array of opportunities. The emphasis on scalable, sustainable, and high-performance infrastructure aligns with Sharon AI’s vision of enabling AI-driven workloads through innovative energy solutions and advanced computing capabilities.

Sharon AI remains committed to supporting initiatives that propel AI growth and infrastructure development, particularly in forward-thinking regions like Texas. With monumental projects like Stargate shaping the future, the AI and data center landscape is on the brink of a revolutionary transformation.

[son1c]

It’s so big brained of you to come up with a reason why some of eggmans bots went rogue while the others didn’t it’s just *chefs kiss* world building my beloved

i <3 inferences

[frank-olivier]

Bayesian Active Exploration: A New Frontier in Artificial Intelligence

The field of artificial intelligence has seen tremendous growth and advancements in recent years, with various techniques and paradigms emerging to tackle complex problems in the field of machine learning, computer vision, and natural language processing. Two of these concepts that have attracted a lot of attention are active inference and Bayesian mechanics. Although both techniques have been researched separately, their synergy has the potential to revolutionize AI by creating more efficient, accurate, and effective systems.

Traditional machine learning algorithms rely on a passive approach, where the system receives data and updates its parameters without actively influencing the data collection process. However, this approach can have limitations, especially in complex and dynamic environments. Active interference, on the other hand, allows AI systems to take an active role in selecting the most informative data points or actions to collect more relevant information. In this way, active inference allows systems to adapt to changing environments, reducing the need for labeled data and improving the efficiency of learning and decision-making.

One of the first milestones in active inference was the development of the "query by committee" algorithm by Freund et al. in 1997. This algorithm used a committee of models to determine the most meaningful data points to capture, laying the foundation for future active learning techniques. Another important milestone was the introduction of "uncertainty sampling" by Lewis and Gale in 1994, which selected data points with the highest uncertainty or ambiguity to capture more information.

Bayesian mechanics, on the other hand, provides a probabilistic framework for reasoning and decision-making under uncertainty. By modeling complex systems using probability distributions, Bayesian mechanics enables AI systems to quantify uncertainty and ambiguity, thereby making more informed decisions when faced with incomplete or noisy data. Bayesian inference, the process of updating the prior distribution using new data, is a powerful tool for learning and decision-making.

One of the first milestones in Bayesian mechanics was the development of Bayes' theorem by Thomas Bayes in 1763. This theorem provided a mathematical framework for updating the probability of a hypothesis based on new evidence. Another important milestone was the introduction of Bayesian networks by Pearl in 1988, which provided a structured approach to modeling complex systems using probability distributions.

While active inference and Bayesian mechanics each have their strengths, combining them has the potential to create a new generation of AI systems that can actively collect informative data and update their probabilistic models to make more informed decisions. The combination of active inference and Bayesian mechanics has numerous applications in AI, including robotics, computer vision, and natural language processing. In robotics, for example, active inference can be used to actively explore the environment, collect more informative data, and improve navigation and decision-making. In computer vision, active inference can be used to actively select the most informative images or viewpoints, improving object recognition or scene understanding.

Timeline:

1763: Bayes' theorem

1988: Bayesian networks

1994: Uncertainty Sampling

1997: Query by Committee algorithm

2017: Deep Bayesian Active Learning

2019: Bayesian Active Exploration

2020: Active Bayesian Inference for Deep Learning

2020: Bayesian Active Learning for Computer Vision

The synergy of active inference and Bayesian mechanics is expected to play a crucial role in shaping the next generation of AI systems. Some possible future developments in this area include:

- Combining active inference and Bayesian mechanics with other AI techniques, such as reinforcement learning and transfer learning, to create more powerful and flexible AI systems.

- Applying the synergy of active inference and Bayesian mechanics to new areas, such as healthcare, finance, and education, to improve decision-making and outcomes.

- Developing new algorithms and techniques that integrate active inference and Bayesian mechanics, such as Bayesian active learning for deep learning and Bayesian active exploration for robotics.

Dr. Sanjeev Namjosh: The Hidden Math Behind All Living Systems - On Active Inference, the Free Energy Principle, and Bayesian Mechanics (Machine Learning Street Talk, October 2024)

Saturday, October 26, 2024

[vox-off]

i plan to spend today eating popcorn and watching redditors fight over whether or not the oblivion remake is a) even real and b) being shadow dropped today, which it apparently is supposed to be

RIP in pieces to the pope or whatever i guess but this is my drama right now

[fossore]

My cousin just started his CS PhD on something like the applications of machine learning for processing the data cities produce so he had all sorts of fun tidbits about people’s driving habits.

Did you know that people drive faster in curved roads than straight roads? And there’s a contingent that thinks there should be no speed limits?

[snarwin]

Perfectly encapsulates the AI art discourse that the complaints on the Coño Culo post are "this is theft because it was made with AI" and not "this is theft because Goku is a copyrighted character."

[peterbordes]

(via AI inference chip startup Groq closes $640M at $2.8B valuation to meet next-gen LPUs demand)

Groq, a leader in fast AI inference, has secured a $640M Series D round at a valuation of $2.8B. The round was led by funds and accounts managed by BlackRock Private Equity Partners with participation from both existing and new investors including Neuberger Berman, Type One Ventures, and strategic investors including Cisco Investments, Global Brain’s KDDI Open Innovation Fund III, and Samsung Catalyst Fund. The unique, vertically integrated Groq AI inference platform has generated skyrocketing demand from developers seeking exceptional speed.

[clouds-of-wings]

I really recommend familiarizing yourself with the way the eBay AI talks in your language if you buy there a lot. It's very useful if you can tell if a product description is generated by a bot because it often makes things up or describes things in exaggeratedly positive terms. Also a bot description is based only on the title and the product info that is already visible above the actual description (and what it thinks it can infer from these), so you know right away the description won't offer any new information and you don't have to read it. I recognize the bot by its writing style now and it helps.

You can familiarize yourself by pretending you want to sell something, writing a fake title and so on, and then just playing around with the AI.

[sharon-ai]

Sharon AI and New Era Helium Partner to Build a 250 MW Net-Zero Data Centre in Texas For more information - https://sharonai.com/blog/sharon-ai-and-new-era-helium-partner-to-build-a-250-mw-net-zero-data-centre-in-texas/

[gigglepuffpixie]

please stop playing with that salami

it's not that interesting

[govindhtech]

AMD ROCm 6.4: Scalable Inference and Smarter AI Workflows

AMD ROCm 6.4: Plug-and-Play Containers, Modular Deployment, and Revolutionary Inference for Scalable AI on AMD Instinct GPUs

Modern AI workloads are larger and more sophisticated, increasing deployment simplicity and performance needs. AMD ROCm 6.4 advances AI and HPC development on AMD Instinct GPUs.

With growing support for leading AI frameworks, optimised containers, and modular infrastructure tools, ROCm software helps customers manage their AI infrastructure, develop faster, and work smarter.

Whether you're managing massive GPU clusters, training multi-billion parameter models, or spreading inference over multi-node clusters, AMD ROCm 6.4 delivers great performance with AMD Instinct GPUs.

This post presents five major AMD ROCm 6.4 improvements that directly address infrastructure teams, model developers, and AI researchers' concerns to enable AI development fast, straightforward, and scalable.

ROCm Training and Inference Containers: Instinct GPU Plug-and-Play AI

Setting up and maintaining ideal training and inference settings takes time, is error-prone, and delays iteration cycles. AMD ROCm 6.4 provides a large set of pre-optimized, ready-to-run training and inference containers for AMD Instinct GPUs.

For low-latency LLM inference, vLLM (Inference Container) supports plug-and-play open models like Gemma 3 (day-0), Llama, Mistral, Cohere, and others.

FP8 support, DeepGEMM, and simultaneous multi-head attention give SGLang (Inference Container) exceptional throughput and efficiency for DeepSeek R1 and agentic processes.

PyTorch (Training Container) makes LLM training on AMD Instinct MI300X GPUs simpler with performance-tuned variations that enable advanced attention strategies. Optimised for FLUX, Llama 2 (70B), and 3.1 (8B, 70B).1-dev.

Training Container Megatron-LM This ROCm-tuned Megatron-LM fork can train large-scale language models like Llama 3.1, Llama 2, and DeepSeek-V2-Lite.

These containers allow AI researchers faster access to turnkey settings for experimentation and model evaluation. Model developers may use pre-tuned support for the most advanced LLMs, including as DeepSeek, Gemma 3, and Llama 3.1, without spending time configuring. These containers also simplify infrastructure teams' maintenance and scale-out by deploying uniformly across development, testing, and production environments.

PyTorch for ROCm Improves: Faster Focus and Training

As training large language models (LLMs) pushes computing and memory limits, ineffective attention strategies can impede iteration and increase infrastructure costs. AMD ROCm 6.4 improves Flex Attention, TopK, and Scaled Dot-Product Attention in PyTorch.

Flex Attention: Outperforms ROCm 6.3 in LLM workloads that need advanced attention algorithms, reducing memory overhead and training time.

TopK: TopK processes are now three times faster, improving inference reaction times without compromising output quality (source).

SDPA: expanded context, smoother inference.

These improvements speed up training, reduce memory overhead, and optimise hardware consumption. As a consequence, model developers can improve bigger models faster, AI researchers can do more tests, and Instinct GPU customers see shorter time-to-train and higher infrastructure ROI.

Upgrades are pre-installed in the ROCm PyTorch container.

AMD Instinct GPU Next-Gen Inference Performance with vLLM and SGLang

Low-latency, high-throughput inference for big language models is difficult, especially when new models develop and deployment pace increases. ROCm 6.4 addresses this problem with AMD Instinct GPU-optimized vLLM and SGLang versions. Due to its strong support for popular models like Grok, DeepSeek R1, Gemma 3, and Llama 3.1 (8B, 70B, and 405B), model developers can deploy real-world inference pipelines with minimal modification or rewrite. AI researchers can get faster time-to-results on large-scale benchmarks. Infrastructure teams can ensure scaled performance, consistency, and reliability with stable, production-ready containers that get weekly updates.

Set an Instinct MI300X throughput record using SGLang and DeepSeek R1.

Day-0 compatibility for Instinct GPU deployment with vLLM Gemma 3.

These technologies create a full-stack inference environment with weekly and bi-weekly development and stable container upgrades.

Smooth Instinct GPU Cluster Management by AMD GPU Operator

Scaling and managing GPU workloads across Kubernetes clusters can cause manual driver updates, operational disruptions, and limited GPU health visibility, which can reduce performance and reliability. With AMD ROCm 6.4, the AMD GPU Operator automates GPU scheduling, driver lifecycle management, and real-time telemetry to optimise cluster operations. This allows AI and HPC administrators to confidently deploy AMD Instinct GPUs in air-gapped and secure environments with full observability, infrastructure teams to upgrade with minimal disruption, and Instinct customers to benefit from increased uptime, lower operational risk, and stronger AI infrastructure.

Some new features are:

Automatic cordon, drain, and reboot for rolling updates.

More support for Ubuntu 22.04/24.04 and Red Hat OpenShift 4.16–4.17 ensures compatibility with modern cloud and enterprise settings.

Device Metrics Exporter for real-time Prometheus health measurements.

The New Instinct GPU Driver Modularises Software

Coupled driver stacks hinder upgrade processes, reduce interoperability, and increase maintenance risk. AMD ROCm 6.4 introduces the modular Instinct GPU Driver, which isolates the kernel driver from ROCm user space.

main benefits,

Infrastructure teams may now upgrade ROCm libraries and drivers separately.

Extended compatibility to 12 months (from 6 months in earlier iterations)

More flexibility in installing ISV software, bare metal, and containers

This simplifies fleet-wide upgrades and reduces the risk of breaking changes, which benefits cloud providers, government agencies, and enterprises with strict SLAs.

AITER for Accelerated Inference Bonus Point

AITER, a high-performance inference library with drop-in, pre-optimized kernels, removes tedious tuning in AMD ROCm 6.4.

Gives:

It can decode 17 times quicker.

14X multi-head focus improvements

Double LLM inference throughput

[raffaellopalandri]

Advanced Methodologies for Algorithmic Bias Detection and Correction

I continue today the description of Algorithmic Bias detection. Photo by Google DeepMind on Pexels.com The pursuit of fairness in algorithmic systems necessitates a deep dive into the mathematical and statistical intricacies of bias. This post will provide just a small glimpse of some of the techniques everyone can use, drawing on concepts from statistical inference, optimization theory, and…

[honysoytquimalpence]

[ramenheim]

#GardenPathSentences

AI has problems re: algorithm training and image ownership, but this image and this image only can stay.

[frank-olivier]

The Predictive Self: Insights from Active Inference and the Free Energy Principle

The concepts of Active Inference and the Free Energy Principle have emerged as pivotal frameworks in understanding the intricacies of agency, whether in biological systems or artificial intelligence. These theories, rooted in the intersection of neuroscience, philosophy, and AI, offer a profound lens through which to examine the fundamental nature of interaction between agents and their environments.

At its core, Active Inference posits that agents engage with their environment in a manner that minimizes surprise, thereby reducing entropy. This process is underpinned by the Free Energy Principle, which suggests that agents strive to optimize the fit between their internal models of the world and the external reality. This optimization is achieved by minimizing the free energy, a proxy for the difference between the agent's predictions and the actual sensory input. The Free Energy Principle thus provides a unifying framework for understanding perception, action, and learning across various scales and systems.

The discussion around Active Inference inevitably touches upon the concepts of agency and free will. By positioning agents as entities that resist entropy, the theory subtly aligns with compatibilist views, where free will is seen as compatible with determinism. This perspective challenges traditional dichotomies, encouraging a more nuanced understanding of agency that transcends the biological-artificial divide. Furthermore, the emphasis on minimizing surprise underscores the predictive nature of the mind, echoing theories that suggest our understanding of the world is fundamentally confabulatory and based on predictive models.

The distinction between observational systems, like ChatGPT, and truly interactive, adaptive systems based on Active Inference, raises crucial questions about machine understanding and human uniqueness. If human beliefs, motives, and desires are, to some extent, confabulatory, what does this imply for our understanding of consciousness and the development of artificial intelligence? The ethical implications are profound, highlighting the need for careful consideration in the creation of autonomous, adaptive AI systems that may increasingly mirror human-like agency.

The interdisciplinary nature of Active Inference suggests a wide array of applications. Clinically, it may inform new approaches to understanding and treating neurological or psychiatric conditions by focusing on the internal models and predictive processes underlying patient experiences. In the realm of AI and robotics, embracing Active Inference could lead to the development of more interactive, adaptive systems that learn and evolve in a more human-like manner.

Thomas Parr: Active Inference (Machine Learning Street Talk, May 2024)

Thursday, November 21, 2024