#Big Data Architecture

[jcmarchi]

Hugging Face partners with Groq for ultra-fast AI model inference

New Post has been published on https://thedigitalinsider.com/hugging-face-partners-with-groq-for-ultra-fast-ai-model-inference/

Hugging Face partners with Groq for ultra-fast AI model inference

Hugging Face has added Groq to its AI model inference providers, bringing lightning-fast processing to the popular model hub.

Speed and efficiency have become increasingly crucial in AI development, with many organisations struggling to balance model performance against rising computational costs.

Rather than using traditional GPUs, Groq has designed chips purpose-built for language models. The company’s Language Processing Unit (LPU) is a specialised chip designed from the ground up to handle the unique computational patterns of language models.

Unlike conventional processors that struggle with the sequential nature of language tasks, Groq’s architecture embraces this characteristic. The result? Dramatically reduced response times and higher throughput for AI applications that need to process text quickly.

Developers can now access numerous popular open-source models through Groq’s infrastructure, including Meta’s Llama 4 and Qwen’s QwQ-32B. This breadth of model support ensures teams aren’t sacrificing capabilities for performance.

Users have multiple ways to incorporate Groq into their workflows, depending on their preferences and existing setups.

For those who already have a relationship with Groq, Hugging Face allows straightforward configuration of personal API keys within account settings. This approach directs requests straight to Groq’s infrastructure while maintaining the familiar Hugging Face interface.

Alternatively, users can opt for a more hands-off experience by letting Hugging Face handle the connection entirely, with charges appearing on their Hugging Face account rather than requiring separate billing relationships.

The integration works seamlessly with Hugging Face’s client libraries for both Python and JavaScript, though the technical details remain refreshingly simple. Even without diving into code, developers can specify Groq as their preferred provider with minimal configuration.

Customers using their own Groq API keys are billed directly through their existing Groq accounts. For those preferring the consolidated approach, Hugging Face passes through the standard provider rates without adding markup, though they note that revenue-sharing agreements may evolve in the future.

Hugging Face even offers a limited inference quota at no cost—though the company naturally encourages upgrading to PRO for those making regular use of these services.

This partnership between Hugging Face and Groq emerges against a backdrop of intensifying competition in AI infrastructure for model inference. As more organisations move from experimentation to production deployment of AI systems, the bottlenecks around inference processing have become increasingly apparent.

What we’re seeing is a natural evolution of the AI ecosystem. First came the race for bigger models, then came the rush to make them practical. Groq represents the latter—making existing models work faster rather than just building larger ones.

For businesses weighing AI deployment options, the addition of Groq to Hugging Face’s provider ecosystem offers another choice in the balance between performance requirements and operational costs.

The significance extends beyond technical considerations. Faster inference means more responsive applications, which translates to better user experiences across countless services now incorporating AI assistance.

Sectors particularly sensitive to response times (e.g. customer service, healthcare diagnostics, financial analysis) stand to benefit from improvements to AI infrastructure that reduces the lag between question and answer.

As AI continues its march into everyday applications, partnerships like this highlight how the technology ecosystem is evolving to address the practical limitations that have historically constrained real-time AI implementation.

(Photo by Michał Mancewicz)

See also: NVIDIA helps Germany lead Europe’s AI manufacturing race

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is co-located with other leading events including Intelligent Automation Conference, BlockX, Digital Transformation Week, and Cyber Security & Cloud Expo.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

[ludopticon]

Welcome to the Ludopticon

We once fought for the right to be forgotten.

Now we fight for the right to be seen, perceived and remembered by people, companies and algorithms.

Michel Foucault described the Panopticon as a system in which individuals self-regulate their behaviour under the constant fear of being watched. However, we have entered a new stage: one that I call the Ludopticon.

This fear, I argue, has now transformed into desire. We do not just accept surveillance; we crave it.

We yearn to be seen, perceived and monetized.

We film, photograph and upload our digital selves, living spaces, possessions and actions; geotagged and labelled for consumption.

Surveillance is no longer solely coercive: it is gamified, incentivised and oftentimes self-inflicted.

[hanasatoblogs]

Big Data and the Internet of Things (IoT): The Power of Analytics

In today’s hyperconnected world, the intersection of the Internet of Things (IoT) and Big Data analytics is reshaping industries, providing businesses with unprecedented insights, and fueling a new wave of innovation. The vast amount of data generated by IoT devices offers immense opportunities to derive actionable insights. By leveraging IoT Big Data solutions, companies can optimize processes, enhance customer experiences, and drive business growth.

This article explores how IoT Big Data analytics, IoT Big Data architecture, and machine learning are transforming industries and providing valuable solutions.

The Explosion of IoT Data

The Internet of Things refers to the network of physical devices connected to the internet, gathering and sharing data. These devices include everything from smart home appliances and wearable health monitors to industrial sensors and autonomous vehicles. According to Statista, the number of IoT-connected devices is projected to reach 30.9 billion by 2025, generating a massive amount of data.

This data deluge presents significant challenges but also immense opportunities for organizations. By implementing IoT Big Data solutions, companies can collect, store, analyze, and act on this vast amount of information to improve decision-making, efficiency, and innovation.

IoT Big Data Analytics: Turning Data Into Insights

One of the most significant advantages of combining IoT with Big Data analytics is the ability to transform raw data into actionable insights. IoT Big Data analytics involves analyzing large volumes of data generated by IoT devices to identify patterns, trends, and anomalies that can inform business decisions.

Real-World Application: In the automotive industry, companies like Tesla use IoT sensors embedded in vehicles to monitor real-time data related to performance, maintenance needs, and driving patterns. This data is then processed through Big Data analytics to improve vehicle performance, anticipate maintenance issues, and even enhance autonomous driving features. Tesla’s ability to leverage IoT Big Data is a key factor in its innovative approach to automotive technology.

Moreover, GE Aviation uses IoT sensors in aircraft engines to monitor real-time performance data. By leveraging Big Data analytics, GE predicts engine failures and schedules proactive maintenance, improving safety and reducing downtime.

IoT Big Data Architecture: The Backbone of Data Processing

To efficiently process and analyze data from millions of IoT devices, businesses need a scalable and robust IoT Big Data architecture. This architecture typically includes:

Data Collection Layer: Sensors and devices collect and transmit data.

Data Ingestion Layer: Middleware solutions or platforms like Apache Kafka are used to ingest data in real-time, handling the large influx of information from various IoT sources.

Data Storage Layer: Data is stored in cloud-based or on-premise databases. Solutions like AWS IoT or Azure IoT are popular choices for storing and managing vast amounts of IoT data.

Data Processing and Analytics Layer: Advanced analytics platforms, such as Hadoop or Apache Spark, process large datasets to extract insights.

Visualization Layer: Insights are presented through dashboards or visualization tools, allowing stakeholders to make informed decisions.

This architecture supports the seamless flow of data from collection to actionable insights, enabling organizations to scale their IoT initiatives.

IoT and Machine Learning: Driving Smarter Solutions

The integration of machine learning with IoT Big Data creates smarter, more predictive systems. Machine learning models analyze the vast datasets generated by IoT devices to detect patterns, learn from them, and predict future outcomes. This combination unlocks powerful IoT Big Data solutions for industries ranging from healthcare to manufacturing.

Practical Example: In healthcare, IoT medical devices such as wearable fitness trackers and smart medical sensors monitor patients’ vitals, including heart rate, blood pressure, and oxygen levels. By feeding this data into machine learning models, healthcare providers can predict potential health risks and intervene early. For instance, machine learning algorithms can detect irregular heart patterns in real-time and alert doctors before a critical event occurs, ultimately saving lives.

In manufacturing, IoT sensors on equipment monitor real-time performance and detect potential failures. By integrating machine learning, manufacturers can predict when machinery is likely to fail and schedule maintenance ahead of time. This proactive approach reduces downtime and increases efficiency.

IoT Big Data Solutions: Real-World Impact

Industries are already reaping the benefits of IoT Big Data solutions, transforming how they operate and deliver value to customers.

Smart Cities: Cities like Barcelona and Singapore have deployed IoT sensors to monitor traffic patterns, optimize waste management, and manage energy consumption. With Big Data analytics, city administrators can improve urban planning and enhance the quality of life for residents. Smart traffic systems use IoT data to reduce congestion, while smart lighting systems adjust brightness based on real-time data to conserve energy.

Retail: IoT sensors in stores can monitor customer behavior, including how long they spend in certain areas or which products they interact with the most. Retailers like Amazon leverage this data to personalize in-store experiences, manage inventory more efficiently, and optimize store layouts. Amazon Go stores, for example, use IoT sensors to track what customers pick up, allowing for a seamless checkout-free shopping experience.

Agriculture: IoT devices in agriculture monitor soil conditions, weather patterns, and crop health. IoT Big Data analytics helps farmers optimize water usage, improve crop yields, and reduce waste. Companies like John Deere use IoT data from smart farming equipment to provide farmers with real-time insights on field conditions, enabling more precise and efficient farming practices.

Overcoming IoT Big Data Challenges

While the potential of IoT Big Data is vast, there are challenges that businesses need to overcome to fully realize its value.

Data Security: With the large volume of sensitive data being collected, organizations must prioritize the security of their IoT Big Data architecture. Ensuring data encryption, secure authentication, and regular vulnerability assessments are essential to safeguarding IoT data.

Data Quality: The sheer amount of data generated by IoT devices can lead to issues with data quality. Companies need to implement systems that filter out irrelevant or redundant data to ensure that only valuable insights are derived.

Scalability: As the number of connected devices grows, so does the complexity of managing IoT Big Data solutions. Businesses need scalable architectures that can handle exponential growth in data while maintaining efficiency.

The Future of IoT and Big Data

The convergence of IoT and Big Data analytics is set to drive significant advancements in many sectors, including healthcare, manufacturing, smart cities, and retail. As IoT devices become more ubiquitous, businesses will increasingly rely on IoT Big Data solutions to make data-driven decisions, improve efficiency, and create personalized experiences.

Looking ahead, the integration of artificial intelligence (AI) and machine learning with IoT will further enhance predictive capabilities, enabling even more accurate forecasting and decision-making. For instance, autonomous vehicles will rely heavily on IoT and Big Data analytics to process vast amounts of real-time data from sensors, allowing for safer and more efficient driving experiences.

Conclusion

The fusion of the Internet of Things and Big Data analytics offers unprecedented opportunities for businesses to harness the power of real-time data and make more informed, timely decisions. By implementing robust IoT Big Data architectures and integrating machine learning models, companies can derive actionable insights that lead to greater operational efficiency, improved customer experiences, and innovation across industries.

As IoT continues to evolve, businesses that invest in the right IoT Big Data solutions will be well-positioned to lead in a data-driven future.

Browse Related Blogs – 

Revolutionize Your Healthcare Strategy with Big Data: What Every CXO Needs to Know

The Power of Customer Journey Mapping: Lessons from Amazon, Starbucks, Netflix and Disney

[aakarshanstar]

Innovative Data Engineering for Strategic Decision-Making

Unlocking the Power of Data: The Role of Data Engineering in Modern Businesses

In today's data-driven world, businesses are increasingly relying on vast amounts of data to make informed decisions, streamline operations, and drive growth. However, the true potential of data can only be harnessed when it is efficiently collected, processed, and analyzed. This is where Data Engineering comes into play—a critical component that forms the backbone of any successful data strategy. At aakarshansedge.com, our Data Engineering services are designed to transform raw data into actionable insights, empowering businesses to thrive in the digital age.

Key Benefits of Our Data Engineering Services Scalability: For scalability in our Data Engineering Services, we ensure that our solutions can seamlessly adapt to increasing data volumes and complexity. Our infrastructure is designed to handle growth efficiently, providing robust performance and flexibility as your data needs evolve. Data Quality: Poor data quality can lead to inaccurate insights and misguided decisions. We implement rigorous data cleaning and validation processes to ensure that your data is accurate, consistent, and trustworthy. Efficiency: In the corporate world, time is of the essence. Our efficient data pipelines and optimized processing techniques minimize latency, allowing you to access and analyze data in real-time. Security and Compliance: With data privacy regulations becoming increasingly stringent, we prioritize security and compliance in all our data engineering projects. We implement robust encryption, access controls, and monitoring systems to protect your data. Cost-Effectiveness: We help you optimize your data storage and processing costs by leveraging cloud platforms and modern data architectures, ensuring you get the most value out of your investment.

Technologies Used in Data Engineering

Big Data Frameworks - The Big Data frameworks at Aakarshan Edge include cutting-edge tools designed for scalable data processing and analytics, such as Apache Hadoop, Apache Spark, and Apache Flink.

Data Warehousing Solutions - Transform your data into actionable insights with our cutting-edge Data Warehousing Solutions, designed for scalability and efficiency at Aakarshan Edge."

Data Integration Tools - Discover top-tier data integration tools at Aakarshan Edge, designed to streamline and enhance your data management processes.

Database Technologies - The website Aakarshan Edge, utilizes advanced database technologies to ensure robust, scalable, and secure data management.

ETL Tools - The website Aakarshan Edge, utilizes cutting-edge ETL (Extract, Transform, Load) tools to streamline data processing and integration, ensuring efficient data management and insights.

Cloud Platforms - Aakarshan Edge offers innovative solutions across leading cloud platforms to enhance scalability and performance for your business.

Data Governance & Quality Tools - Implement robust Data Governance and Quality Tools to ensure the accuracy, consistency, and security of your data assets.

Data Visualization Tools - Transform complex data into clear, actionable insights with our advanced data visualization tools. From interactive dashboards to customizable charts, we empower your business to make data-driven decisions with ease.

Programming Languages - The website Aakarshan Edge, uses a combination of programming languages including HTML, CSS, JavaScript, and potentially server-side languages like PHP or Python.

Machine Learning Libraries - The website Aakarshan Edge, features cutting-edge machine learning libraries to enhance data analytics and predictive modeling.

Why Choose Aakarshan Edge for Data Engineering?

At Aakarshan Edge, we understand that every business is unique, and so are its data challenges. Our approach to data engineering Solutions is highly customized, focusing on understanding your specific needs and delivering solutions that align with your business objectives. Our team of experienced data engineers is well-versed in the latest technologies and best practices, ensuring that your data infrastructure is future-proof and capable of driving innovation.

Conclusion

our Data Engineering Services at Aakarshan Edge are designed to empower your business with robust data solutions that drive efficiency and innovation. By leveraging advanced technologies and tailored strategies, we ensure that your data infrastructure is not only scalable but also aligned with your strategic goals. Partner with us to transform your data into a powerful asset that enhances decision-making and fuels growth.

Contact us (+91-8860691214) (E-Mail: sales@akarshans.com)

[sbscglobal]

Welcome to the digital era, where data reigns as the new currency.

In modern information technology, the term “Big Data” has surged to the forefront, embodying the exponential growth and availability of data in today’s digital age. This influx of data encompasses vast volumes, generated at unprecedented speeds and with diverse varieties, presenting both challenges and opportunities across industries worldwide.

To unlock the true potential of big data, businesses need to address several critical areas like #BigDataCollection and #DataIntegration, #DataStorage and Management, #DataAnalysis and #DataAnalytics, #DataPrivacy and #DataSecurity, Innovation and Product Development, Operational Efficiency and Cost Optimization. Here at SBSC we recognize the transformative power of #bigdata and empower businesses to unlock its potential through a comprehensive suite of services: #DataStrategy and #Consultation: SBSC’s Tailored advisory services help businesses define their Big Data goals, develop a roadmap, and align data initiatives with strategic objectives.

#DataArchitecture and #DataIntegration: We Design and implementation of scalable, robust data architectures that support data ingestion, storage, and integration from diverse sources. #DataWarehousing and Management: SBSC provides Solutions for setting up data warehouses or data lakes, including management of structured and unstructured data, ensuring accessibility and security. Data Analytics and Business Intelligence: Advanced analytics capabilities leveraging machine learning, AI algorithms, and statistical models to derive actionable insights and support decision-making.

#DataVisualization and Reporting: Creation of intuitive dashboards and reports that visualize key insights and performance metrics, enabling stakeholders to interpret data effectively. #CloudServices and Infrastructure: Leveraging #cloudplatforms for scalability, flexibility, and cost-effectiveness in managing Big Data environments, including migration and optimization services Continuous Improvement and Adaptation: Establishment of feedback loops and metrics to measure the impact of Big Data initiatives, fostering a culture of continuous improvement and adaptation.

By offering a comprehensive suite of services in these areas, SBSC helps businesses to harness the power of Big Data to drive innovation, improve operational efficiency, enhance customer experiences, and achieve sustainable growth in today’s competitive landscape

Contact SBSC to know the right services you need for your Business

Email: info@sbsc.com Website:https://www.sbsc.com

[nitor-infotech]

Understanding Database Sharding

Imagine you have a library with thousands of books, and finding a specific one becomes time-consuming. To speed things up, you decide to split the collection into smaller sections based on genres, making it quicker to locate any book.  

Similarly, Database Sharding divides a large database into smaller, more manageable pieces to improve performance and scalability. 

Fig: Database Sharding Banner Image 

What is a Sharding Database 

Database sharding is like storing a huge database across several machines. Imagine one server trying to handle all the data—it can only do so much before it starts slowing down. By splitting the data into smaller chunks, or shards, and storing these across multiple servers, we can manage and process large amounts of data more efficiently. 

As an application grows, more users and data can turn the database into a bottleneck, slowing everything down and frustrating users. Sharding also helps by allowing parallel processing of these smaller datasets, keeping things running smoothly even as demand increases. 

Scaling Techniques in Database Sharding 

Scaling database sharding involves several techniques to ensure efficient management and distribution of data. Here are some key methods: 

Horizontal Partitioning 

This technique involves splitting the data across multiple servers based on a sharding key, such as user ID or geographic region. Each server, or shard, contains a subset of the overall data. This approach allows the system to scale out by adding more servers, thereby distributing the load and improving performance and reliability. 

Vertical Partitioning 

This technique divides the database into smaller tables, each stored on different servers. Each server handles a specific aspect of the application, such as user profiles, transactions, or product details. By separating the data based on functionality, vertical partitioning can improve query performance and make it easier to manage and scale specific parts of the application independently. 

Range-Based Sharding 

Distributes data management based on a continuous range of values. For example, user IDs 1-1000 on one shard, and 1001-2000 on another. 

Hash-Based Sharding 

Uses a hash function on the sharding key to evenly distribute data across shards. This helps avoid uneven data distribution. 

Directory-Based Sharding 

Maintains a lookup table or directory that maps each data item to its corresponding shard. This allows flexible and dynamic distribution of data. 

Each technique has its advantages and is chosen based on the specific needs and growth patterns of the application. 

Benefits of Database Sharding 

Database sharding offers several benefits: 

Improved Performance: By distributing the data across multiple servers, each server handles a smaller subset of the data, reducing the load and improving query response times. 

Scalability: Sharding allows horizontal scaling, meaning you can add more servers to handle the increased load, making it easier to scale the database as the application grows. 

Increased Availability: With data distributed across multiple servers, the system can continue to operate even if one shard fails. This redundancy enhances the overall availability and reliability of the application. 

Efficient Resource Utilization: Different shards can be optimized for specific workloads, allowing better use of hardware resources. For instance, high-traffic shards can be allocated more resources, while less busy shards use fewer resources. 

Reduced Maintenance: Smaller databases are easier to back up, restore, and maintain. Sharding breaks down the database into more manageable pieces, simplifying administrative tasks. 

Factors to consider before Sharding 

Before deciding to shard your database, consider the following factors: 

Database Size: Sharding is typically suitable for large databases that have outgrown the capacity of a single server. 

Traffic Patterns: If your database experiences uneven traffic patterns, sharding can help balance the load. 

Growth Projections: If significant future scaling is anticipated, sharding can be a beneficial strategy. 

Complexity: Sharding introduces additional complexity to your database architecture and requires careful planning and ongoing maintenance. 

Cost: Sharding can be costly due to the need for extra hardware resources and infrastructure to support multiple servers. 

So, database sharding offers both advantages and challenges, and it is important to determine if it aligns with your application’s requirements. 

To know more about database management, reach out to us at Nitor Infotech. 

[francescolelli]

Phd or Postdoc in Switzerland for International Students: On Swiss Government Excellence Scholarships

This is a short preview of the article: Do you have a fresh master or PhD and are you considering Phd or Postdoc in Switzerland? The Swiss Government Excellence Scholarship offers young researchers from around the world who have completed a master’s degree or PhD the opportunity to start or continue their research careers in S

If you like it consider checking out the full version of the post at: Phd or Postdoc in Switzerland for International Students: On Swiss Government Excellence Scholarships

If you are looking for ideas for tweet or re-blog this post you may want to consider the following hashtags:

Hashtags: #BigData, #CloudComputing, #DigitalDecisionMaking, #DistributedSystems, #Fellowship, #HumanBehaviourInformatics, #InternetOfThings, #IoT, #PhD, #PostDoc, #ServiceOrientedArchitecture, #Swiss, #Switzerland, #VirtualReality

The Hashtags of the Categories are: #BigData, #CloudComputing, #InternetofThings, #Job, #Job/Fellowship, #MachineLearning, #Programming, #Research, #SoftwareEngineering

Phd or Postdoc in Switzerland for International Students: On Swiss Government Excellence Scholarships is available at the following link: https://francescolelli.info/job/phd-or-postdoc-in-switzerland-for-international-students-on-swiss-government-excellence-scholarships/ You will find more information, stories, examples, data, opinions and scientific papers as part of a collection of articles about Information Management, Computer Science, Economics, Finance and More.

The title of the full article is: Phd or Postdoc in Switzerland for International Students: On Swiss Government Excellence Scholarships

It belong to the following categories: Big Data, Cloud Computing, Internet of Things, Job, Job/Fellowship, Machine Learning, Programming, Research, Software Engineering

The most relevant keywords are: Big Data, Cloud Computing, Digital Decision Making, Distributed Systems, fellowship, Human Behaviour Informatics, internet of things, IoT, PhD, Post-Doc, Service Oriented Architecture, Swiss, Switzerland, Virtual Reality

It has been published by Francesco Lelli at Francesco Lelli a blog about Information Management, Computer Science, Finance, Economics and nearby ideas and opinions

Do you have a fresh master or PhD and are you considering Phd or Postdoc in Switzerland? The Swiss Government Excellence Scholarship offers young researchers from around the world who have completed a master’s degree or PhD the opportunity to start or continue their research careers in S

Hope you will find it interesting and that it will help you in your journey

Do you have a fresh master or PhD and are you considering Phd or Postdoc in Switzerland? The Swiss Government Excellence Scholarship offers young researchers from around the world who have completed a master’s degree or PhD the opportunity to start or continue their research careers in Switzerland. The scholarship supports research endeavors for a…

[rajaniesh]

Scaling Your Data Mesh Architecture for maximum efficiency and interoperability

[mobio-solutions]

Delving into the essentials of Data Warehouse Architecture. 📈 It's more than just tech talk; it's about building the backbone of our digital future. Every piece of data tells a story, and in the vast universe of information, Data Warehouses are the libraries where these stories are kept. 📚✨ Stay connected for insights that could transform your approach to data. 🔍 For more, scan the QR code and read our article.

[technicalfika]

What is the difference between Data Scientist and Data Engineers ?

In today’s data-driven world, organizations harness the power of data to gain valuable insights, make informed decisions, and drive innovation. Two key players in this data-centric landscape are data scientists and data engineers. Although their roles are closely related, each possesses unique skills and responsibilities that contribute to the successful extraction and utilization of data. In…

[not-terezi-pyrope]

Often when I post an AI-neutral or AI-positive take on an anti-AI post I get blocked, so I wanted to make my own post to share my thoughts on "Nightshade", the new adversarial data poisoning attack that the Glaze people have come out with.

I've read the paper and here are my takeaways:

Firstly, this is not necessarily or primarily a tool for artists to "coat" their images like Glaze; in fact, Nightshade works best when applied to sort of carefully selected "archetypal" images, ideally ones that were already generated using generative AI using a prompt for the generic concept to be attacked (which is what the authors did in their paper). Also, the image has to be explicitly paired with a specific text caption optimized to have the most impact, which would make it pretty annoying for individual artists to deploy.

While the intent of Nightshade is to have maximum impact with minimal data poisoning, in order to attack a large model there would have to be many thousands of samples in the training data. Obviously if you have a webpage that you created specifically to host a massive gallery poisoned images, that can be fairly easily blacklisted, so you'd have to have a lot of patience and resources in order to hide these enough so they proliferate into the training datasets of major models.

The main use case for this as suggested by the authors is to protect specific copyrights. The example they use is that of Disney specifically releasing a lot of poisoned images of Mickey Mouse to prevent people generating art of him. As a large company like Disney would be more likely to have the resources to seed Nightshade images at scale, this sounds like the most plausible large scale use case for me, even if web artists could crowdsource some sort of similar generic campaign.

Either way, the optimal use case of "large organization repeatedly using generative AI models to create images, then running through another resource heavy AI model to corrupt them, then hiding them on the open web, to protect specific concepts and copyrights" doesn't sound like the big win for freedom of expression that people are going to pretend it is. This is the case for a lot of discussion around AI and I wish people would stop flagwaving for corporate copyright protections, but whatever.

The panic about AI resource use in terms of power/water is mostly bunk (AI training is done once per large model, and in terms of industrial production processes, using a single airliner flight's worth of carbon output for an industrial model that can then be used indefinitely to do useful work seems like a small fry in comparison to all the other nonsense that humanity wastes power on). However, given that deploying this at scale would be a huge compute sink, it's ironic to see anti-AI activists for that is a talking point hyping this up so much.

In terms of actual attack effectiveness; like Glaze, this once again relies on analysis of the feature space of current public models such as Stable Diffusion. This means that effectiveness is reduced on other models with differing architectures and training sets. However, also like Glaze, it looks like the overall "world feature space" that generative models fit to is generalisable enough that this attack will work across models.

That means that if this does get deployed at scale, it could definitely fuck with a lot of current systems. That said, once again, it'd likely have a bigger effect on indie and open source generation projects than the massive corporate monoliths who are probably working to secure proprietary data sets, like I believe Adobe Firefly did. I don't like how these attacks concentrate the power up.

The generalisation of the attack doesn't mean that this can't be defended against, but it does mean that you'd likely need to invest in bespoke measures; e.g. specifically training a detector on a large dataset of Nightshade poison in order to filter them out, spending more time and labour curating your input dataset, or designing radically different architectures that don't produce a comparably similar virtual feature space. I.e. the effect of this being used at scale wouldn't eliminate "AI art", but it could potentially cause a headache for people all around and limit accessibility for hobbyists (although presumably curated datasets would trickle down eventually).

All in all a bit of a dick move that will make things harder for people in general, but I suppose that's the point, and what people who want to deploy this at scale are aiming for. I suppose with public data scraping that sort of thing is fair game I guess.

Additionally, since making my first reply I've had a look at their website:

Used responsibly, Nightshade can help deter model trainers who disregard copyrights, opt-out lists, and do-not-scrape/robots.txt directives. It does not rely on the kindness of model trainers, but instead associates a small incremental price on each piece of data scraped and trained without authorization. Nightshade's goal is not to break models, but to increase the cost of training on unlicensed data, such that licensing images from their creators becomes a viable alternative.

Once again we see that the intended impact of Nightshade is not to eliminate generative AI but to make it infeasible for models to be created and trained by without a corporate money-bag to pay licensing fees for guaranteed clean data. I generally feel that this focuses power upwards and is overall a bad move. If anything, this sort of model, where only large corporations can create and control AI tools, will do nothing to help counter the economic displacement without worker protection that is the real issue with AI systems deployment, but will exacerbate the problem of the benefits of those systems being more constrained to said large corporations.

Kinda sucks how that gets pushed through by lying to small artists about the importance of copyright law for their own small-scale works (ignoring the fact that processing derived metadata from web images is pretty damn clearly a fair use application).

[jcmarchi]

Magistral: Mistral AI challenges big tech with reasoning model

New Post has been published on https://thedigitalinsider.com/magistral-mistral-ai-challenges-big-tech-with-reasoning-model/

Magistral: Mistral AI challenges big tech with reasoning model

Mistral AI has pulled back the curtain on Magistral, their first model specifically built for reasoning tasks.

Magistral arrives in two flavours: a 24B parameter open-source version called Magistral Small that anyone can tinker with, and a beefier enterprise edition, Magistral Medium, aimed at commercial applications where advanced reasoning capabilities matter most.

“The best human thinking isn’t linear—it weaves through logic, insight, uncertainty, and discovery,” explains Mistral AI.

That’s a fair point, existing models often struggle with the messy, non-linear way humans actually think through problems. I’ve tested numerous reasoning models and they typically suffer from three key limitations: they lack depth in specialised domains, their thinking process is frustratingly opaque, and they perform inconsistently across different languages.

Mistral AI’s real-world reasoning for professionals

For professionals who’ve been hesitant to trust AI with complex tasks, Magistral might change some minds.

Legal eagles, finance folks, healthcare professionals and government workers will appreciate the model’s ability to show its work. All conclusions can be traced back through logical steps—crucial when you’re operating in regulated environments where “because the AI said so” simply doesn’t cut it.

Software developers haven’t been forgotten either. Magistral claims to shine at the kind of structured thinking that makes for better project planning, architecture design, and data engineering. Having struggled with some models that produce plausible-sounding but flawed technical solutions, I’m keen to see if Magistral’s reasoning capabilities deliver on this front.

Mistral claims their reasoning model excels at creative tasks too. The company reports that Magistral is “an excellent creative companion” for writing and storytelling, capable of producing both coherent narratives and – when called for – more experimental content. This versatility suggests we’re moving beyond the era of having separate models for creative versus logical tasks.

What separates Magistral from the rest?

What separates Magistral from run-of-the-mill language models is transparency. Rather than simply spitting out answers from a black box, it reveals its thinking process in a way users can follow and verify.

This matters enormously in professional contexts. A lawyer doesn’t just want a contract clause suggestion; they need to understand the legal reasoning behind it. A doctor can’t blindly trust a diagnostic suggestion without seeing the clinical logic. By making its reasoning traceable, Magistral could help bridge the trust gap that’s held back AI adoption in high-stakes fields.

Having spoken with non-English AI developers, I’ve heard consistent frustration about how reasoning capabilities drop off dramatically outside English. Magistral appears to tackle this head-on with robust multilingual support, allowing professionals to reason in their preferred language without performance penalties.

This isn’t just about convenience; it’s about equity and access. As countries increasingly implement AI regulations requiring localised solutions, tools that reason effectively across languages will have a significant advantage over English-centric competitors.

[embedded content]

Getting your hands on Magistral

For those wanting to experiment, Magistral Small is available now under the Apache 2.0 licence via Hugging Face. Those interested in the more powerful Medium version can test a preview through Mistral’s Le Chat interface or via their API platform.

[embedded content]

Enterprise users looking for deployment options can find Magistral Medium on Amazon SageMaker, with IBM WatsonX, Azure, and Google Cloud Marketplace implementations coming soon.

As the initial excitement around general-purpose chatbots begins to wane, the market is hungry for specialised AI tools that excel at specific professional tasks. By focusing on transparent reasoning for domain experts, Mistral has carved out a potentially valuable niche.

Founded just last year by alumni from DeepMind and Meta AI, Mistral has moved at breakneck speed to establish itself as Europe’s AI champion. They’ve consistently punched above their weight, creating models that compete with offerings from companies many times their size.

As organisations increasingly demand AI that can explain itself – particularly in Europe where the AI Act will require transparency – Magistral’s focus on showing its reasoning process feels particularly timely.

(Image by Stephane)

See also: Tackling hallucinations: MIT spinout teaches AI to admit when it’s clueless

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is co-located with other leading events including Intelligent Automation Conference, BlockX, Digital Transformation Week, and Cyber Security & Cloud Expo.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

[mariacallous]

The so-called Department of Government Efficiency (DOGE) is starting to put together a team to migrate the Social Security Administration’s (SSA) computer systems entirely off one of its oldest programming languages in a matter of months, potentially putting the integrity of the system—and the benefits on which tens of millions of Americans rely—at risk.

The project is being organized by Elon Musk lieutenant Steve Davis, multiple sources who were not given permission to talk to the media tell WIRED, and aims to migrate all SSA systems off COBOL, one of the first common business-oriented programming languages, and onto a more modern replacement like Java within a scheduled tight timeframe of a few months.

Under any circumstances, a migration of this size and scale would be a massive undertaking, experts tell WIRED, but the expedited deadline runs the risk of obstructing payments to the more than 65 million people in the US currently receiving Social Security benefits.

“Of course, one of the big risks is not underpayment or overpayment per se; [it’s also] not paying someone at all and not knowing about it. The invisible errors and omissions,” an SSA technologist tells WIRED.

The Social Security Administration did not immediately reply to WIRED’s request for comment.

SSA has been under increasing scrutiny from president Donald Trump’s administration. In February, Musk took aim at SSA, falsely claiming that the agency was rife with fraud. Specifically, Musk pointed to data he allegedly pulled from the system that showed 150-year-olds in the US were receiving benefits, something that isn’t actually happening. Over the last few weeks, following significant cuts to the agency by DOGE, SSA has suffered frequent website crashes and long wait times over the phone, The Washington Post reported this week.

This proposed migration isn’t the first time SSA has tried to move away from COBOL: In 2017, SSA announced a plan to receive hundreds of millions in funding to replace its core systems. The agency predicted that it would take around five years to modernize these systems. Because of the coronavirus pandemic in 2020, the agency pivoted away from this work to focus on more public-facing projects.

Like many legacy government IT systems, SSA systems contain code written in COBOL, a programming language created in part in the 1950s by computing pioneer Grace Hopper. The Defense Department essentially pressured private industry to use COBOL soon after its creation, spurring widespread adoption and making it one of the most widely used languages for mainframes, or computer systems that process and store large amounts of data quickly, by the 1970s. (At least one DOD-related website praising Hopper's accomplishments is no longer active, likely following the Trump administration’s DEI purge of military acknowledgements.)

As recently as 2016, SSA’s infrastructure contained more than 60 million lines of code written in COBOL, with millions more written in other legacy coding languages, the agency’s Office of the Inspector General found. In fact, SSA’s core programmatic systems and architecture haven’t been “substantially” updated since the 1980s when the agency developed its own database system called MADAM, or the Master Data Access Method, which was written in COBOL and Assembler, according to SSA’s 2017 modernization plan.

SSA’s core “logic” is also written largely in COBOL. This is the code that issues social security numbers, manages payments, and even calculates the total amount beneficiaries should receive for different services, a former senior SSA technologist who worked in the office of the chief information officer says. Even minor changes could result in cascading failures across programs.

“If you weren't worried about a whole bunch of people not getting benefits or getting the wrong benefits, or getting the wrong entitlements, or having to wait ages, then sure go ahead,” says Dan Hon, principal of Very Little Gravitas, a technology strategy consultancy that helps government modernize services, about completing such a migration in a short timeframe.

It’s unclear when exactly the code migration would start. A recent document circulated amongst SSA staff laying out the agency’s priorities through May does not mention it, instead naming other priorities like terminating “non-essential contracts” and adopting artificial intelligence to “augment” administrative and technical writing.

Earlier this month, WIRED reported that at least 10 DOGE operatives were currently working within SSA, including a number of young and inexperienced engineers like Luke Farritor and Ethan Shaotran. At the time, sources told WIRED that the DOGE operatives would focus on how people identify themselves to access their benefits online.

Sources within SSA expect the project to begin in earnest once DOGE identifies and marks remaining beneficiaries as deceased and connecting disparate agency databases. In a Thursday morning court filing, an affidavit from SSA acting administrator Leland Dudek said that at least two DOGE operatives are currently working on a project formally called the “Are You Alive Project,” targeting what these operatives believe to be improper payments and fraud within the agency’s system by calling individual beneficiaries. The agency is currently battling for sweeping access to SSA’s systems in court to finish this work. (Again, 150-year-olds are not collecting social security benefits. That specific age was likely a quirk of COBOL. It doesn’t include a date type, so dates are often coded to a specific reference point—May 20, 1875, the date of an international standards-setting conference held in Paris, known as the Convention du Mètre.)

In order to migrate all COBOL code into a more modern language within a few months, DOGE would likely need to employ some form of generative artificial intelligence to help translate the millions of lines of code, sources tell WIRED. “DOGE thinks if they can say they got rid of all the COBOL in months, then their way is the right way, and we all just suck for not breaking shit,” says the SSA technologist.

DOGE would also need to develop tests to ensure the new system’s outputs match the previous one. It would be difficult to resolve all of the possible edge cases over the course of several years, let alone months, adds the SSA technologist.

“This is an environment that is held together with bail wire and duct tape,” the former senior SSA technologist working in the office of the chief information officer tells WIRED. “The leaders need to understand that they’re dealing with a house of cards or Jenga. If they start pulling pieces out, which they’ve already stated they’re doing, things can break.”

[1920secretsociety]

Let's talk about the prison scene:

This has always been my favourite scene in Ghostbusters. I know it may seem a bit weird to find this scene amazing but bare with me.

This scene is their most vulnerable moment, they are in a state of limbo at whether they are going to jail or walk free. No one is in control here, they have no idea what's going to happen, meaning we see the true and hidden characteristics of each Ghostbuster.

Egon

Egon mostly proves to be everything we thought. He is logical and intelligent, he seems to be the only one who has done any form of in-depth research on Dana's case. He found the architect's name (Ivo Shandor) and all his history while working full time as a Ghostbuster, inventing their equipment and keeping an eye on the storage facility's data, reinforcing the preconception that Egon is a workaholic and incredibly smart. However, it's also a moment you see a new characteristic - confidence. When Egon tells Venkman, Ray and Winston what is happening, Egon is only expecting them to listen, when everyone else in the cell decides to listen you see that Egon is shocked. It's obvious that he doesn't expect anyone to listen but when he comes to terms with the fact the other prisoners are listening, Egon gains confidence. When stating facts about Shandor, Egon becomes increasingly confident and he even stands up at the end to prove the severity of the situation. This is never seen before as Egon is usually subdued, quiet and rarely ever uses body language, and in this whole scene he is completely different. This shows that Egon has the characteristic of confident and that he can command a presence, which is shown by the fact that everyone starts listening to him. He is a confident individual when it comes to his science and facts, especially when explaining them and this is completely shown here. Egon is not always the reserved scientist we thought, he has confidence but just expresses it differently.

Ray

Ray is shown to be intelligent in multiple different ways. Ray is completely involved in the investigation of Dana's apartment as he does his part of the research and learns the foundation that the architecture is weird, it's a conductor and these are all facts Egon builds on. This shows that Ray is a intelligent individual who gives facts that are as valuable as Egon's, and along with this a different type of intelligence emerges. In this scene, Ray is the only one who realises that Venkman has been lying the whole time about studying. This took a lot of intelligence to work out as you usually trust people, especially ones you have known for a long time and come from a credential background. The fact Ray works it out shows that he has a deep knowledge of his friends, proving that he has amazing intelligence when it comes to understanding people. However, it also shows Ray's bravey. Making an accusation that big could have backfired as Venkman may have become deeply offended, if he actually had studied. Therefore, it shows that Ray is an individual who is intelligent and incredibly brave as questioning people's credentials, could result in a massive argument or a falling out. This whole scene shows that Ray has an amazing perception and intelligence of people as he deduces Venkman's lies immediately and that he has confidence in each of his analysis, as he said that Venkman was lying with full confidence, despite the only legitimate evidence being a feeling.

Winston

Winston is showed to be intelligent and logical. In this scene he seems to be the only one whose genuinely concerned about going to jail, as he's the only one who shouts at the guard, trying to get out. The rest are not really bothered, they're more concerned about sharing information on Gozer, but as much as Winston believes in it he understands that being in jail is the more important problem, as they can't stop Gozer if they're locked up. He is also the only one who realises that the reality of Gozer coming is just going to look insane in court, as to the outside world ghosts are not really considered real, never mind a ghost attack. All of this reveals that Winston is intelligent and logical, he is a realist and has an amazing perception of the outside world and social situations. He's as intelligent as Egon but in a completely different way, his intelligence lies in his knowledge of the real world, and this gives him a vital role within Ghostbusters as without Winston's intelligence of the outside world they would have no real idea of how to navigate it.

Venkman

Venkman is shown to be helpful and caring. After Egon is finished explaining that Gozer is coming back all the prisoners are still around him. This would make Egon uneasy as he hates social situations and he has no more science or facts to make him feel confident as he's finished explaining. Venkman knows how uneasy Egon is and helps as he starts singing. He does this to draw attention away from Egon and while singing he forces everyone back, and he keeps going they are all far away from Egon. It's a small thing but it shows how much he really does care about Egon and all of the Ghostbusters. Venkman obviously understands them all on a deep level as it took him about a second to realise that Egon was uncomfortable. This all shows that despite all his faults Venkman is helpful and more importantly caring, he will do anything to insure that the rest of the his colleagues are happy and secure, even if it means embarrassing himself.

This is why I absolutely love this scene. We see glimpses of their personalities that we never see in any other situation. It's the most honest they all are, it shows their character development and more importantly gives us all a million more reasons to love them.

[cognitivejustice]

“Sometimes it gets so hot, I can’t think straight,” said Chunara, sporting a black smartwatch that contrasts sharply with her colourful bangles and sari.

Chunara is one of 204 residents of Vanzara Vas given the smartwatches for a year-long study to find out how heat affects vulnerable communities around the world. The watches measure heart rate and pulse and track sleep, and participants get weekly blood pressure checks.

Data collector Komal Parmar, right, talks with Sapnaben Chunara to get heat related information in Ahmedabad, India.AP Photo/Ajit Solanki

Researchers also painted some roofs with reflective paint to reduce indoor heat and will compare them to homes without so-called cool roofs using indoor heat sensors. Along with the smartwatches, this will help them understand how much cool roofs can help poor households deal with India’s scorching summers.

A man applies reflective paint on the roof of a house to reduce indoor heat in Ahmedabad, India.AP Photo/Ajit Solanki

Chunara, whose home didn't get a cool roof, said she's happy to participate by wearing the watch, confident the results will help her family, too.

"They might paint my roof as well, and they might be able to do something that helps all of us in this area cope with the heat better,” Chunara said.

An increasingly hot planet, due largely to burning fossil fuels such as coal and gas that release carbon dioxide and other greenhouse gases, means already hot regions are getting even worse.

A 2023 study estimated that if the global mean temperature continues to rise to just under 2 degrees Celsius, there would be a 370 per cent rise in heat-related deaths around the world, and most would happen in South and Southeast Asia and Africa.

“This is a big concern, and it also shows the heat divide” between the poor and wealthy, said Abhiyant Tiwari, a climate expert with the Natural Resources Defence Council and part of the group conducting the research in Ahmedabad.

In the summer of 2010, the city witnessed nearly 1,300 excess deaths — how many more people died than would be expected — which experts found were most likely due to high temperatures.

Following the 2010 tragedy, city officials, with help from public health and heat experts, devised an action plan to warn citizens when the heat is at dangerous levels and prepare city hospitals to respond rapidly to heat-related illness. The plan has been replicated across India and other parts of South Asia.

I studied design in Ahmedabad's National Institute of Design. Reading this helps explain the design of our campus, architecture that emphasized air circulation and natural cooling. Mind you, I was there umpteen million years ago in 1989-1990.

[nitor-infotech]

Discover how Database sharding can transform your application's performance by distributing data across multiple servers in our latest blog. With insights into key sharding techniques, you'll further learn how to implement sharding effectively and avoid common pitfalls.

As you move forward, this blog will help you dive into real-life use cases to understand how sharding can optimize data management. Lastly, you'll get the most important factors to consider before sharding your database and learning to navigate the complexities of database management.