#Deep & Reinforcement Learning

[jcmarchi]

RAGEN: AI framework tackles LLM agent instability

New Post has been published on https://thedigitalinsider.com/ragen-ai-framework-tackles-llm-agent-instability/

RAGEN: AI framework tackles LLM agent instability

Researchers have introduced RAGEN, an AI framework designed to counter LLM agent instability when handling complex situations.

Training these AI agents presents significant hurdles, particularly when decisions span multiple steps and involve unpredictable feedback from the environment. While reinforcement learning (RL) has shown promise in static tasks like solving maths problems or generating code, its application to dynamic, multi-turn agent training has been less explored.   

Addressing this gap, a collaborative team from institutions including Northwestern University, Stanford University, Microsoft, and New York University has proposed StarPO (State-Thinking-Actions-Reward Policy Optimisation).

StarPO offers a generalised approach for training agents at the trajectory level (i.e. it optimises the entire sequence of interactions, not just individual actions.)

Accompanying this is RAGEN, a modular system built to implement StarPO. This enables the training and evaluation of LLM agents, particularly focusing on their reasoning capabilities under RL. RAGEN provides the necessary infrastructure for rollouts, reward assignment, and optimisation within multi-turn, stochastic (randomly determined) environments.

Minimalist environments, maximum insight

To isolate the core learning challenges from confounding factors like extensive pre-existing knowledge or task-specific engineering, the researchers tested LLMs using RAGEN in three deliberately minimalistic, controllable symbolic gaming environments:   

Bandit: A single-turn, stochastic task testing risk-sensitive symbolic reasoning. The agent chooses between options (like ‘Phoenix’ or ‘Dragon’ arms) with different, initially unknown, reward profiles.

Sokoban: A multi-turn, deterministic puzzle requiring foresight and planning, as actions (pushing boxes) are irreversible.

Frozen Lake: A multi-turn, stochastic grid navigation task where movement attempts can randomly fail, demanding planning under uncertainty.

These environments allow for clear analysis of how agents learn decision-making policies purely through interaction.   

Key findings: Stability, rollouts, and reasoning

The study yielded three significant findings concerning the training of self-evolving LLM agents:

The ‘Echo Trap’ and the need for stability

A recurring problem observed during multi-turn RL training was dubbed the “Echo Trap”. Agents would initially improve but then suffer performance collapse, overfitting to locally rewarded reasoning patterns. 

This was marked by collapsing reward variance, falling entropy (a measure of randomness/exploration), and sudden spikes in gradients (indicating training instability). Early signs included drops in reward standard deviation and output entropy.   

To combat this, the team developed StarPO-S, a stabilised version of the framework. StarPO-S incorporates:   

Variance-based trajectory filtering: Focusing training on task instances where the agent’s behaviour shows higher uncertainty (higher reward variance), discarding low-variance, less informative rollouts. This improved stability and efficiency.   

Critic incorporation: Using methods like PPO (Proximal Policy Optimisation), which employ a ‘critic’ to estimate value, generally showed better stability than critic-free methods like GRPO (Group Relative Policy Optimisation) in most tests.   

Decoupled clipping and KL removal: Techniques adapted from other research (DAPO) involving asymmetric clipping (allowing more aggressive learning from positive rewards) and removing KL divergence penalties (encouraging exploration) further boosted stability and performance.   

StarPO-S consistently delayed collapse and improved final task performance compared to vanilla StarPO.   

Rollout quality is crucial

The characteristics of the ‘rollouts’ (simulated interaction trajectories used for training) significantly impact learning. Key factors identified include:   

Task diversity: Training with a diverse set of initial states (prompts), but with multiple responses generated per prompt, aids generalisation. The sweet spot seemed to be moderate diversity enabling contrast between different outcomes in similar scenarios.   

Interaction granularity: Allowing multiple actions per turn (around 5-6 proved optimal) enables better planning within a fixed turn limit, without introducing the noise associated with excessively long action sequences.   

Rollout frequency: Using fresh, up-to-date rollouts that reflect the agent’s current policy is vital. More frequent sampling (approaching an ‘online’ setting) leads to faster convergence and better generalisation by reducing policy-data mismatch.

Maintaining freshness, alongside appropriate action budgets and task diversity, is key for stable training.   

Reasoning requires careful reward design

Simply prompting models to ‘think’ doesn’t guarantee meaningful reasoning emerges, especially in multi-turn tasks. The study found:

Reasoning traces helped generalisation in the simpler, single-turn Bandit task, even when symbolic cues conflicted with rewards.   

In multi-turn tasks like Sokoban, reasoning benefits were limited, and the length of ‘thinking’ segments consistently declined during training. Agents often regressed to direct action selection or produced “hallucinated reasoning” if rewards only tracked task success, revealing a “mismatch between thoughts and environment states.”

This suggests that standard trajectory-level rewards (often sparse and outcome-based) are insufficient. 

“Without fine-grained, reasoning-aware reward signals, agent reasoning hardly emerge[s] through multi-turn RL.”

The researchers propose that future work should explore rewards that explicitly evaluate the quality of intermediate reasoning steps, perhaps using format-based penalties or rewarding explanation quality, rather than just final outcomes.   

RAGEN and StarPO: A step towards self-evolving AI

The RAGEN system and StarPO framework represent a step towards training LLM agents that can reason and adapt through interaction in complex, unpredictable environments.

This research highlights the unique stability challenges posed by multi-turn RL and offers concrete strategies – like StarPO-S’s filtering and stabilisation techniques – to mitigate them. It also underscores the critical role of rollout generation strategies and the need for more sophisticated reward mechanisms to cultivate genuine reasoning, rather than superficial strategies or hallucinations.

While acknowledging limitations – including the need to test on larger models and optimise for domains without easily verifiable rewards – the work opens “a scalable and principled path for building AI systems” in areas demanding complex interaction and verifiable outcomes, such as theorem proving, software engineering, and scientific discovery.

(Image by Gerd Altmann)

See also: How does AI judge? Anthropic studies the values of Claude

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is co-located with other leading events including Intelligent Automation Conference, BlockX, Digital Transformation Week, and Cyber Security & Cloud Expo.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

[pardomuansitanggang]

Artificial Intelligence Researcher, Research & Publication Technology Specialist, Data Analyst, National Editor & Reviewer, Applied Researcher, PARDOMUANSITANGGANG.COM – Berikut adalah gambaran tentang lima peran yang Anda sebutkan, bersama dengan keterampilan dan strategi pengembangan karier yang dapat membantu Anda sukses di setiap peran tersebut: 1. Artificial Intelligence (AI) Researcher Deskripsi Peran: Seorang peneliti AI berfokus pada mengembangkan dan memajukan teknologi kecerdasan buatan, termasuk teknik machine learning, deep learning, pemrosesan bahasa alami (NLP), dan penglihatan komputer. Mereka bertanggung jawab untuk merancang algoritma, melakukan eksperimen, dan mempublikasikan temuan ilmiah di konferensi atau jurnal bereputasi. Keterampilan yang Diperlukan: Matematika dan Statistik: Pemahaman kuat tentang teori probabilitas, statistik, dan kalkulus sangat penting. Pemrograman: Penguasaan bahasa seperti Python, R, C++, dan framework AI seperti TensorFlow, Keras, dan PyTorch. Algoritma Machine Learning: Pengetahuan tentang algoritma seperti regresi, SVM, CNN, RNN, GAN, reinforcement learning. Pemahaman Teori AI: Kemampuan memahami model teoretis yang digunakan untuk AI, termasuk etika AI dan dampak sosialnya. Strategi Pengembangan: Ikuti Kursus Lanjutan: Selalu perbarui pengetahuan dengan mengikuti kursus online atau sertifikasi dari platform seperti Coursera, edX, atau Udacity. Publikasi Riset: Aktif dalam mempublikasikan karya di jurnal dan konferensi AI top seperti NeurIPS, ICML, atau CVPR. Bergabung dengan Komunitas AI: Terlibat dalam komunitas open-source seperti GitHub, Kaggle, atau arXiv untuk berbagi dan belajar dari sesama peneliti. 2. Research & Publication Technology Specialist Deskripsi Peran: Spesialis ini bekerja pada implementasi dan pemeliharaan teknologi untuk mendukung proses penelitian dan publikasi. Mereka membantu peneliti menggunakan alat teknologi untuk mengumpulkan data, menganalisis, dan mempublikasikan hasil penelitian. Keterampilan yang Diperlukan: Teknologi Manajemen Pengetahuan: Memahami alat-alat seperti reference management (Mendeley, EndNote) dan sistem submission jurnal. Data Analysis Tools: Kemampuan menggunakan perangkat analisis data seperti SPSS, MATLAB, atau R. Publikasi Digital: Familiar dengan teknologi untuk publikasi digital, termasuk manajemen metadata dan akses terbuka (open access). Strategi Pengembangan: Pelajari Sistem Jurnal Akademik: Pahami alur kerja manajemen jurnal, dari penyerahan artikel hingga peer review. Fokus pada Open Access: Memahami tren dalam publikasi open access dan repositori penelitian. Networking: Terlibat dalam konferensi dan acara yang berfokus pada teknologi riset dan publikasi. 3. Data Analyst Deskripsi Peran: Data analyst bertugas mengumpulkan, mengolah, dan menganalisis data untuk memberikan wawasan yang dapat mendukung pengambilan keputusan. Mereka sering bekerja dengan set data besar (big data) dan menggunakan alat analitik untuk menemukan tren dan pola. Keterampilan yang Diperlukan: Pemrograman: Menguasai bahasa seperti Python, R, dan SQL untuk analisis data. Statistik: Pemahaman kuat tentang statistik dasar dan inferensial. Alat Visualisasi Data: Menguasai alat visualisasi data seperti Tableau, Power BI, atau D3.js. Komunikasi: Kemampuan untuk menyampaikan temuan data dengan cara yang mudah dipahami oleh pemangku kepentingan non-teknis. Strategi Pengembangan: Proyek Praktis: Ikuti proyek atau kompetisi data science di platform seperti Kaggle untuk mendapatkan pengalaman langsung. Pelajari Alat Terbaru: Selalu ikuti perkembangan terbaru di teknologi big data, termasuk Hadoop, Spark, dan alat cloud seperti AWS atau Google Cloud. Bangun Portofolio: Buat portofolio online dari proyek-proyek yang sudah dikerjakan untuk menunjukkan keterampilan Anda kepada calon pemberi kerja. 4. National Editor & Reviewer Deskripsi Peran: Editor nasional dan reviewer bertanggung jawab untuk mengevaluasi manuskrip dan artikel yang diajukan ke jurnal, memastikan kualitas ilmiah dan keaslian karya sebelum diterbitk...

[learnershub101]

6 Udemy Paid Courses for Free with Certification (Limited Time for Enrollment)

1. Computer Science MetaBootcamp: Beginner to Intermediate 2024

Learn the Basics of Programming, Cyber Systems, Python, Cryptography, Web Development, Cybersecurity, Networking, ML & more!

What you'll learn

 Finally, understand how computers and all cyber systems work at a basic level.

 Take a journey through all the layers of cyber systems beginning from electrical signals at the bottom up to the algorithms in your head.

 Understand programming concepts such as Model View Controller (MVC), bit manipulation, and much more.

 A holistic understanding of basics such as variables, loops, conditionals and much more.

 Learn scripting and automation through Python, the top programming language of today used in Web Development, Machine Learning, and Data Science.

Understand how the internet works and other networking concepts.

Learn web technologies such as HTML, CSS, and Javascript.

 Take your first step into the field of cybersecurity by completing a Capture the Flag challenge on the popular TryHackMe platform.

 Prepare for jobs or internships by practicing common leetcode problems that are commonly asked in tech interviews today.

 Become language agnostic and migrate easily from 1 language and framework to another by converting your Python project to a web page using HTML, CSS, and JS.

 Follow along with interesting storylines and characters that reinforce the concepts learned in the course.

 Speed up to 5 semesters' worth of a Computer Science and Engineering Degree on demand and quickly recognize your strengths, weaknesses, and interests.

 Be able to easily differentiate and understand the basics of each of the giant sub-fields within CSE.

 Understand clearly and exactly what a CS student walks like, thinks like, and acts like!

 Identify a path for yourself within the tech-infused careers of tomorrow!

Take This Course

2. Python Wizardry: Master the Path from Start to Finish

Python A-Z: Your Path to Becoming a Programming Maestro: Machine Learning, Python Programming, Web Development, ChatGPT

What you'll learn

 Mastery from Scratch: Master Python fundamentals with hands-on exercises.

 Advanced Syntax: Dive deep into Python's powerful features and syntax.

Efficient Debugging: Learn techniques to troubleshoot and fix errors.

 OOP Proficiency: Become skilled in Object-Oriented Programming concepts.

 Data Manipulation: Excel in data handling, manipulation, and analysis.

Web Development: Build dynamic websites using Python frameworks.

GUI Creation: Craft graphical user interfaces for applications.

Database Integration: Integrate databases and manage data seamlessly.

Automation Magic: Harness Python's automation capabilities for tasks.

 Testing & Optimization: Ensure code quality and enhance performance.

Project Management: Plan, develop, and deploy Python projects.

 Problem-Solving Prowess: Develop logical thinking for real-world challenges.

Take This Course

3. ChatGPT Python Alchemy: Elevate Your Coding with AI

Unleash the Potential of ChatGPT and Python Fusion for Next-Level Coding Proficiency | Natural Language Processing

What you'll learn

Fuse Python with AI for powerful applications.

Integrate ChatGPT seamlessly in Python.

Develop advanced AI-driven Python projects.

Elevate user interactions through AI.

Optimize code using AI insights.

Master NLP techniques in Python.

Create AI-powered chatbots and assistants.

Implement ethical AI practices in coding.

Enhance problem-solving with AI capabilities.

Apply AI to analyze and process data.

Craft dynamic AI-enhanced applications.

Master the synergy of Python and AI creativity.

Take This Course

4. JavaScript Wizardry: The Basics Unveiled

Master the Power of JavaScript in under 10 hrs: Mastering the Inner Workings for Javascript: ChatGPT : HTML: CSS

What you'll learn

 Solid Understanding of JavaScript Fundamentals: Learners will develop a strong foundation in JavaScript, including variables, data types, control structures etc

 Proficient DOM Manipulation: Learners will gain expertise in interacting with the Document Object Model (DOM), manipulating elements, handling events, etc.

 Advanced JavaScript Concepts: Learners will explore advanced topics such as closures, prototypes, scope, execution context, and asynchronous programming,     etc.

 Mastery of JavaScript Libraries and Frameworks: Learners will become proficient in popular JavaScript libraries and frameworks, such as jQuery, React, etc.

 Effective Error Handling and Debugging: Learners will learn techniques to debug and troubleshoot JavaScript code effectively, including error handling.

 Efficient Data Manipulation with Arrays and Objects: Learners will acquire skills in working with arrays and objects, including manipulating, filtering, etc.

 Practical Understanding of JavaScript APIs: Learners will explore various JavaScript APIs, such as the Fetch API for making HTTP requests, etc.

 Introduction to Server-Side JavaScript: Learners will be introduced to server-side JavaScript using technologies like Nodejs, Expressjs, or MongoDB,     etc.

 Security Best Practices in JavaScript: Learners will understand common security vulnerabilities in JavaScript and learn best practices to mitigate risks etc.

Take This Course

5. ChatGPT Secrets | ChatGPT Mastery | ChatGPT Masterclass

ChatGPT 101 | The Beginner To Expert Guide To All You Need To Know About CHATGPT Inside & Out!

What you'll learn

 Develop a deeper understanding of how natural language processing (NLP) and deep learning algorithms can be used to create conversational agents.

Design and develop a conversational agent using CHATGPT.

 Evaluate and optimize the performance of a CHATGPT-based conversational agent.

 Utilize best practices in the development of a CHATGPT-based conversational agent.

 Implement methods for data collection and annotation to train a CHATGPT-based conversational agent.

 Understand the ethical implications of using conversational agents and their applications.

 Explore the benefits and limitations of using CHATGPT for conversational agent development.

 Identify potential areas for further research and development for CHATGPT.

Take This Course

6. Facebook Ads Secrets | Facebook Marketing: The Untold

Learn Facebook Ads on a high level | Understand the power of Facebook Marketing | Unleash the combination of ChatGPT

What you'll learn

 Understand the fundamentals of Facebook advertising and marketing, including ad types, targeting options, and bidding strategies.

 Learn how to create effective Facebook ad campaigns that generate leads, drive traffic, and increase sales for your business.

 Gain knowledge of advanced topics such as retargeting, lookalike audiences, and custom audiences to increase the effectiveness of your Facebook advertising.

 Create compelling ad copy and creative that resonates with your target audience, and aligns with your business goals.

 Measure and optimize ad performance using Facebook's powerful analytics tools, and identify areas for improvement.

 Understand how to use Facebook as a powerful tool for brand awareness, customer engagement, and customer retention.

 Gain a deep understanding of Facebook's algorithm, and how it impacts the performance of your ad campaigns.

 Apply best practices for Facebook advertising and marketing, and learn from real-world case studies and examples of successful Facebook campaigns.

Take This Course

[fuzzyperfectionengineer]

Chemistry 5070 and Chemistry 0620 book to get A* in Exams

Unlock Chemistry Success with our Comprehensive Guide: CAIE O Level Chemistry 5070 and IGCSE Chemistry 0620 Complete Book! 🧪📚

Hello Chemistry Enthusiasts,

Embark on a journey of academic excellence with our dynamic resource, the CAIE O Level Chemistry 5070 and IGCSE Chemistry 0620 Complete Book. Crafted to serve as your go-to guide for quick preparation and a crash course to secure that coveted A* in your exams. This comprehensive guide is your key to mastering the intricacies of Chemistry effortlessly.

Get Your Chemistry Guide Now!

Why Choose Our Chemistry Guide?

In the ever-evolving landscape of Chemistry education, having a reliable and comprehensive guide is crucial. Our Chemistry guide stands out for the following reasons:

📚 Extensive Coverage: Covering the entire syllabus of CAIE O Level Chemistry 5070 and IGCSE Chemistry 0620, this guide leaves no topic untouched. It's your one-stop solution for a quick yet thorough revision.

🎓 Crash Course Format: Designed as a crash course, this book enables quick understanding and retention of essential concepts, making it ideal for last-minute preparation.

🔍 Easy Accessibility: The guide is available in PDF format, providing ease of access for students. Download, study, and revise on your preferred device.

🌐 Global Relevance: Whether you're studying in the USA, the UK, or any other part of the world, our guide is designed to cater to the needs of students globally.

Key Features of the Chemistry Guide:

🔬 Interactive PDF Format: Our guide is presented in a visually appealing and interactive PDF format, making your learning experience engaging and effective.

🌈 In-depth Conceptual Understanding: Each chapter is crafted to ensure a deep understanding of concepts, with clear explanations and illustrative examples.

💡 Practice Questions and Answers: Reinforce your learning with a plethora of practice questions and detailed answers to assess your understanding.

📖 Syllabus Integration: Aligned with the latest syllabus, the guide ensures that you cover all the necessary topics for your upcoming exams.

Unlock the Secrets to Chemistry Success - Download Now!

Explore the world of Chemistry with our guide that covers a wide array of topics, ensuring you are well-prepared and confident when facing your exams.

Download the Chemistry Guide Now!

What Educators, Students, Parents, and Tutors are Saying:

Educators:

Ms. Anderson, Chemistry Teacher

Location: New York, USA

Review: "I've been using this guide to supplement my classroom teachings, and the results are outstanding. It simplifies complex concepts, making it easier for students to grasp. Highly recommended! ⭐⭐⭐⭐⭐"

Dr. Patel, Chemistry Professor

Location: London, UK

Review: "As a Chemistry professor, I appreciate resources that bridge the gap between theory and application. This guide does just that. It's a valuable asset for both students and educators. ⭐⭐⭐⭐⭐"

Students:

Emily Thompson, High School Student

Location: Sydney, Australia

Feedback: "I was struggling with Chemistry, but this guide turned things around. The interactive format and clear explanations helped me understand complex topics. I'm more confident now! ⭐⭐⭐⭐⭐"

Rajesh Kumar, IGCSE Student

Location: Mumbai, India

Feedback: "This guide is a lifesaver! It's concise, covers everything I need, and the practice questions are a game-changer. My grades have improved significantly. Thank you! ⭐⭐⭐⭐⭐"

Parents:

Mrs. Rodriguez, Parent of a High Schooler

Location: Mexico City, Mexico

Testimonial: "I bought this guide for my daughter, and I can see the positive impact it has had on her studies. It's well-organized, and she finds it very helpful. Highly recommend it to other parents! ⭐⭐⭐⭐⭐"

Mr. Smith, Guardian of an O Level Student

Location: Toronto, Canada

Testimonial: "This guide is a valuable resource for parents too! It helps me support my child's learning at home. The clear explanations make Chemistry more approachable. Fantastic tool! ⭐⭐⭐⭐⭐"

Tutors:

Tutor Kate, Private Chemistry Tutor

Location: Cape Town, South Africa

Recommendation: "I recommend this guide to all my students. It complements my teaching style and serves as an excellent resource for extra practice. A must-have for tutors! ⭐⭐⭐⭐⭐"

Professor Lee, Online Chemistry Tutor

Location: Seoul, South Korea

Recommendation: "This guide enhances the online learning experience. It's visually appealing, interactive, and provides a comprehensive understanding of Chemistry. Great for virtual classrooms! ⭐⭐⭐⭐⭐"

Why Invest in Our Chemistry Guide?

⭐ Quick Preparation: Ideal for last-minute revision and crash course preparation.

⭐ High Success Rate: Students using our guide have consistently achieved top grades.

⭐ Conceptual Clarity: Enjoy a clear understanding of complex Chemistry topics.

⭐ Accessible Anywhere: Download the PDF and study on your preferred device.

Join the Chemistry Revolution - Your Path to A Awaits!*

Invest in your academic success and unlock the doors to a brighter future. Download our CAIE O Level Chemistry 5070 and IGCSE Chemistry 0620 Complete Book now!

Download Now!

Join Our Community:

Connect with fellow Chemistry enthusiasts, share insights, and stay updated on the latest Chemistry trends. Join our Telegram Community for real-time discussions, additional resources, and support.

Embark on Your Chemistry Journey Today!

Get Your Chemistry Guide Now! 🌟🧪

chemistry-5070-book

o-level-chemistry-5070-book-pdf

chemistry-5070-book-pdf

chemistry-5070-book-pdf-download

chemistry-books-for-high-school

a-chemist-has-70-ml-of-a-50-methane-solution

e-chem-ep50

chemistry-o-level-green-book

general-chemistry-5th-edition

o-level-chemistry-book-in-pakistan

chemistry-o-level-new-book

chemistry-5070-notes-pdf

o-level-chemistry-book-price

o-level-chemistry-practical-book

old-chemistry-textbooks

vintage-chemistry-books

w-chemistry

o-level-chemistry-book-2023

chemistry-book-2-answers

4-chemistry

6-chemistry

6-chemistry-in-biology

7th-grade-chemistry-textbook-pdf

7-chemistries

[heycoach-super30]

HeyCoach's Expert Pathway: Advanced Training in Data Structures, Algorithms & System Design

Embark on a transformative journey in the tech world with HeyCoach's specialized course in Data Structures, Algorithms, and System Design. Designed for aspiring tech professionals aiming to crack interviews at top-tier companies like Google, Meta, and Apple, our course offers an unmatched learning experience.

HeyCoach Key Features:

Comprehensive Coverage: Delve deep into the essentials of Data Structures and Algorithms. From basic to advanced concepts, our curriculum is tailored to empower you with the knowledge required to solve complex problems efficiently.

System Design Expertise: Learn both low-level and high-level system design principles. Our course prepares you to design scalable, robust systems, a critical skill sought by leading tech firms.

Industry-Relevant Curriculum: Updated regularly to reflect the latest trends and technologies, ensuring you're always ahead of the curve.

Expert Instruction: Our trainers are industry stalwarts from tech behemoths, offering invaluable insights and real-world perspectives.

Interactive Learning Experience: Benefit from LIVE classes, hands-on projects, and personalized feedback to reinforce your learning.

Interview Preparation: Specialized modules focused on tech interview strategies, including mock interviews and problem-solving sessions.

Career Support: Guidance on effective hiring discussions and strategic offer negotiations to help you land your dream job.

Why Choose HeyCoach?

Proven Track Record: Our 'Super 30' program has facilitated entry into renowned tech giants, boasting a stellar peak CTC of 98 LPA.

Practical Application: Emphasis on real-world project development to ensure theoretical knowledge is effectively applied in practical scenarios.

Expansive Network: Collaborations with over 350 companies, enhancing your networking opportunities in the tech industry.

Enroll Today: https://heycoach.in/super30?modal=true&utm_source=seo&utm_medium=imagesubmissions&utm_campaign=imagesubmissions

Realize the benefits of this streamlined program by getting placed in a tier 1 company of your choice.

Contact us today to boost your career to the next level: Email: team@heycoach.in Mobile: +91-9353912654 For more information, visit our website: https://heycoach.in/super30 Follow us for updates and insights: Facebook : https://www.facebook.com/heycoach.in LinkedIn : https://www.linkedin.com/company/hey-coach-in/ YouTube : https://www.youtube.com/watch?v=kUhFWM_Uo0k Instagram: https://www.instagram.com/heycoach.in/

[theawakenedstate]

The Ultimate Manifestation Guide: How to Manifest Made Simple

So you want to Manifest and you’re not even sure where to start? Well then, you’ve come to the RIGHT place. I am so in love with manifestation for so many reasons but the main one is asking you: How can you not be obsessed with co-creating your own reality? Once you start to know how to do it, am I right, or am I right? So Let’s talk about Manifestation from the Beginning:

WHAT THE HECK IS MANIFESTATION?

What is manifestation? In short, it is about learning that you can consciously create the life you desire. Manifestation is you learning that you hold the ability to bring something from the non-physical into physical form with your Thoughts, vibration, feelings and aligned action. It is the process of you claiming from the quantum reality what you desire to have and then magnetizing it in through your beingness. Manifesting is quite simply learning how to own your magnetic nature of who you are and then start using that to your personal advantage. In laymans terms ‘you want something” – You Set an Intention and Use principles of manifestation to consciously attract it into your life. Here’s my “Let’s sit down for coffee and talk about it” version, Check out the link below.

READ THE FULL ARTICLE HERE WHAT IS MANIFESTING?

The Four-Step Formula to Manifest ANYTHING you want

Now that you know what manifestation is, it’s time to PRACTICE Manifestation. Manifestation is not something that we try once, say “that’s cute” and then don’t do anything else! It’s a consistent practice to learn to manifest. Here’s my classic 4 step Formula that you can start to apply to anything for persistent & consistent manifestation success. Pay Attention. Take notes and Try it for yourself!

READ THE 4 STEPS TO MANIFEST FORMULA

Now What Area do you want to Practice Manifesting? Pick your Poison 😉

LOVE: Manifesting Love and Attracting in a Relationship

A lot of people ask questions around: manifesting a text, manifesting a specific person, manifesting a relationship, manifesting a twin flame or simply just being more magnetic when it comes to relationships.

From my own personal experience, These are my top 3 tips when it comes to Manifesting Love and what to immediately focus on when it comes to amping up your attraction power in your personal relationships.

Warning – If you actually apply these 3 tips you will definitely start to experience a shift 😉

3 TIPS TO MANIFEST LOVE & ATTRACTION

CONFIDENCE: Give Yourself a Self-esteem Boost & Manifest Self-Confidence

Do you have a lot of self-doubt or self-criticism when it comes to taking action? You need to work on manifesting better confidence in yourself. This is exactly how you can manifest a Self-esteem Boost and Build Self- Confidence. 

In this Video I cover: + How to Easily Give yourself a Self-esteem boost with the power of manifestation + How to fact check your language so you catch yourself! + The EASIEST way to build your self-confidence muscle immediately

3 TIPS TO MANIFEST SELF- CONFIDENCE

HEALTH: Manifest Divine Health & Empower your Healing Journey

Have a hard time with health and illness, frequently find yourself getting sick? Have some misconceptions around energy work and Struggling to heal yourself ? do you want a different conversation on Energy Healing that goes deep into the Mind-Body Connection and Energetic Patterns? 

Whether you are suffering from an illness or just simply want to reinforce good health, this is a great conversation on what you need to be focused on to manifest divine health in your life.

What we discuss in this video:

+ What really causes illness my personal perspective

+ How Energetics play a massive role in Healing ourselves from the inside out.

+ Learn ways you may be sabotaging your own Health and how to immediately shift this!

3 STEPS TO MANIFEST DIVINE HEALTH

DREAM CAREER: Manifest Your Dream Job, Career or Business

How do I manifest my Dream Career, my Dream Business, or my Dream Job? A Question I’ve asked myself a million times.  I imagine if you’re reading this article, you’ve felt the same. Well it Starts with a Solid Foundation of your Career Blueprint.

Here’s the thing, We won’t manifest something that is out of integrity with who we are, this is often a misaligned goal if we feel this way. This is why we need a solid foundation to start manifesting a new career and set up our vision for success!

i’m sharing my Top 3 steps on how you can use manifesting principles with attracting in your Dream Job or Career starting today. 

3 TIPS FOR MANIFESTING YOUR DREAM JOB

ABUNDANCE MINDSET: Manifest an Abundance Mindset For Ultimate Success

ABUNDANCE IS YOUR NATURAL STATE OF CONSCIOUSNESS. I deeply believe Abundance is your Natural Birthright, the problem? We’ve been training our minds to see otherwise. some of us? for most of our lives. If you feel abundance is just about money, It’s time to look beyond the surface level into the energetics at play here. Let’s Explore How to Manifest an Abundance Mindset. Sharing: + My Top 3 OMG WORKS FAST Tips on Abundance Mindset

+ How to Easily Lock into Abundance and Ditch Lack for good(it’s easier than it sounds!)

+ Why Abundance Mindset is CRUCIAL to Manifestation Let me know what you feel from this week’s episode.

3 STEPS FOR AN ABUNDANCE MINDSET

GOOD HABITS: How to Manifest & Develop Good Habits with LOA Principles

All of your Life is literally made as a string of habits that you have personally installed, & chosen on repeat. So when it comes to understanding Manifestation, we must realize that we can always change our Habits. We can replace bad habits with good habits. We can actively shift mindset habits that drastically don’t serve us like overthinking or negative self-talk and change these with repetition into true good habits. So How can we develop & manifest good habits using the law of attraction? here’s my top 3 steps to immediately help.

3 STEPS TO MANIFESTING GOOD HABITS

FUN & MANIFESTING GAMES FOR YOUR KIDS: Have FUN with Manifesting & Learn to Manifest with your Kids

How can we have more FUN Manifesting with the Law of attraction?

Manifestation is not complicated, it is meant to be FUN, Playful and Enjoyable. That’s why Awaken your Power to Manifest my new book on amazon is made for you to have fun and practice manifesting- Its supposed to be fun 🙂 Sometimes it is essential to remind yourself of this, especially as an adult. If you’re NOT having fun manifesting, then there’s something going on – that you’re missing – having FUN and detaching from the seriousness of it so you release the outcomes 😉 Here’s how to Manifest having more Fun with the Law of Attraction and bonus how to manifest with your kids. Enjoy!

HAVE FUN WITH MANIFESTING GAMES

SELF-LOVE & CARE: Manifest more Self Love and Care

When it comes to Self-worth, we honestly need to be looking at Manifesting Self-love for ourselves.

Self-worth is created from immense Self Love, nurture and Care. MENTALLY, Physically, and Emotionally.  It carries the roots of ALL our relationships whether romantic or not.  But, the journey must start with you.  So if you’re looking to increase your Self-worth, Work on your self-care rituals or just have better self-love, 

these may be the 3 tips that set you over the edge and elevate your worth for good   Enjoy The conversation in this week’s video: 

+ The unconventional way you can immediately shift your self-worth + How to easily link Manifestation principles with self-love and care to help you think differently + Action steps you can do immediately to create better habits for self-care

3 TIPS TO MANIFEST SELF LOVE

FITNESS: Manifest Better Fitness for Losing weight & Consistent Routine with these Law of Attraction Lacks

Try these 3 Law of attraction hacks for your Fitness routine! One of My Ultimate Hacks for Staying Committed to my Fitness is Combing MINDSET with my fitness Practice. If you’re wanting to increase your consistency, make fitness a habit or simply want to lose weight, you will love the 3 steps i’m about to share. Tune in to the video for the full download. Let me know which Step most speaks to you – Do You use mindset with your fitness routine? share with me in the comments!

3 LAW OF ATTRACTION HACKS FOR FITNESS

MONEY: Manifest Money & Develop your Relationship with Money Consciousness

Are you wanting a More Positive Relationship with your Money Mindset? Then Look no further than this episode On my top 3 tips on manifesting money. Money is simply energy! That’s all it is. It is a Tool and Resource we actively use in our day to day living. However, it’s also simply energy. It is energy that we packed on a TON of meaning, Beliefs, Stigmas, and Interactions in our daily habits. If you’re not looking at your money mindset, I suggest you take some time to do so, it can blow your mind. 

This is why in our final How to Manifest series. I’m going to teach you my top 3 steps on Creating a kickass Money relationship so you can start to have an empowering money mindset with practice. 

Let’s face it, money can get a bad rap for all the conditioning that society has placed on it. 

That’s why I wanted to keep this SIMPLE. 

This week’s video I want to drastically simplify it for you with THREE easy steps you can do immediately to create a kick-ass Money Mindset.  Let’s Get Started

Manifest Money with these 3 Steps

______________

Enjoy the How to Manifest Series! Want more where that came from? Check out my New FREE Training How to Master your Mindset with my Top 5 Manifesting Secrets that you can start to do Immediately.

https://www.theawakenedstate.net/the-ultimate-manifest-guide/

The Ultimate Manifestation Guide: How to Manifest Made Simple

So you want to Manifest and you’re not even sure where to start? Well then, you’ve come to the RIGHT place. I am so in love with manifestation for so many reasons but the main one is asking you: How can you not be obsessed with co-creating your own reality? Once you start to know how to do it, am […]

[crazygalore]

MICHAEL CORLEONE ( THE GODFATHER TRILOGY ) NSFW ALPHABET

Disclaimer: My portrayal of Michael Corleone is almost exclusively movie-based. I have read the book and respect it for what it represents, but I have a preference for movie Michael, since I first watched the film, and only read the book years later. That being said, I will selectively borrow elements from the novel here and there, if and when I see fit.

A = Aftercare (What they’re like after sex)

After having lost all hope of ever finding love again after having suffered through countless tragedies and betrayals, it is safe to assume that Michael treats you right in all aspects of your life together. So, after sex, he will draw you a hot bath and gently help you wash up and dry off afterwards. Then, he will tenderly tuck you into bed, bring you any snacks and beverages you may be craving, and afterwards he will lay down next to you and hold you against him until you fall asleep with your head on his chest.

B = Body part (Their favourite body part of theirs and also their partner’s)

He’s survived all of these years by keeping his eyes open and his mind clear, so he is quite fond of his keen and sharp gaze. He’s a bit of a sapiosexual, so while he thinks you are the sexiest and most attractive person in this world, it was your mind and your spirit that made him fall in love with you in the beginning. So, to this very day, having a stimulating conversation with you amps up his libido like nothing else.

C = Cum (Anything to do with cum basically… I’m a disgusting person)

He’s a traditionalist at heart, so he prefers finishing deep inside of you, although he will mark you with his seed if that was something you enjoyed. You would have to ask him to do it, though.

D = Dirty Secret (Pretty self explanatory, a dirty secret of theirs)

He loves seeing you wear his clothes - especially his shirts - because he thinks of it as a reinforcement of his claim on you. Although he is not one of those overly jealous or extremely possessive partners, the sight of you enveloped in his garments always manages to reassure him that you are his and his alone, to love and cherish until the day he dies. 

E = Experience (How experienced are they? Do they know what they’re doing?)

Surprisingly, Michael has only slept with three women in the past - namely a former girlfriend and his two former wives. But although his experience is relatively limited, he’s been doing it long enough to know exactly how to drive his partner insane with desire and pleasure.

F = Favourite Position (This goes without saying. Will probably include a visual)

Usually it’s either missionary or doggy style, depending on the mood. On one hand, he likes being able to hold you in his arms and look into your eyes as he pounds into you - but on the other hand, the sight of your pretty ass and his girth disappearing inside of you drives him absolutely bonkers. Once in a while, however, when he is particularly exhausted, Michael will just sit back and relax as you ride him to completion. 

G = Goofy (Are they more serious in the moment, or are they humorous, etc)

Normally, he is very passionate and serious in bed, because making love to you is one of the few times when he can show you his more vulnerable and romantic side. That being said, realistically speaking sex is a very accident-prone activity in general, and some mishaps are funnier than others - and he is comfortable enough around you to crack an apologetic smile when that happens. 

H = Hair (How well groomed are they, does the carpet match the drapes, etc.)

With a military background and his current position as a don, Michael is a very clean and tidy person, and he keeps himself well groomed and trimmed down there. It is not his habit to shave completely, however, unless you expressed your preference for it.

I = Intimacy (How are they during the moment, romantic aspect…)

Michael is a man hardened by the lifestyle he had no choice but embrace. He has learned to guard his heart, smother his conscience and use his reason to make educated and oftentimes ruthless decisions for the sake of his family’s safety and prosperity. But you are his sanctuary, and in your arms he feels loved, chrished and nurtured - and he feels the need to reciprocate your gestures of affection and show you his vulnerable and romantic side. So sex with Michael is always an intimate, passionate and intense experience.

J = Jack Off (Masturbation headcanon)

He doesn’t do it that often - if at all. When it happens, it’s always in the shower, and he sees it as nothing more but instant gratification for his body’s biological functions - nothing more. It only ever happens if you two happen to be away from each other, for example during his solo business trips.

K = Kink (One or more of their kinks)

A food kink, because nothing gets him going quite like licking whipped cream and frosting off your gorgeous body. Light bondage and domination - but nothing too extreme, as his intention is simply to bring you pleasure and not to demean you. Edging, orgasm denial and mild spanking, but nothing more hardcore than that. As previously stated, he loves seeing you wearing his shirts, and he has fucked you countless times while you were still garbed in them. Also a mild and barely there breeding kink.

L = Location (Favourite places to do the do)

He’s a very private individual, so only in the comfort and safety of your home, usually in your bedroom or your shared bathroom. If he is 100% sure nobody will accidentally walk in on you too, he will definitely fuck you on the kitchen counter, and put that food kink of his to good use.

M = Motivation (What turns them on, gets them going)

As a sapiosexual ( a person who finds intelligence to be a sexually attractive quality in others ), having a long and stimulating debate with you is an instant turn-on. Other than that, you are gorgeous and you are all his, and your very presence fills him with desire. So, as long as the two of you are alone, without the risk of being interrupted, he’s game. But if you’re ever not feeling like it, he won’t pressure you into having sex with him, nor will he act grumpy because of it. You are his beloved, and he respect and adores you at all times.

N = NO (Something they wouldn’t do, turn offs)

He will never harm or demean you in any way. So anything too extreme and damaging is out of the question, no matter how much you might beg him to change his mind.

O = Oral (Preference in giving or receiving, skill, etc)

The sight of you, kneeling before him, with your pretty lips wrapped around his girth is heaven for Michael. There are things that you can do with your wicked tongue that drive him completely crazy with need, and he oftentimes finds himself thrusting against your talented mouth. That being said, he reciprocates the gesture every single time, and he is very skilled at it. With a flick of his tongue in the right spot, and a well placed suckle, he can bring you to completion in no time - and he usually isn’t satisfied unless he’s given you several earth-shattering orgasms using only his mouth and deft fingers.

P = Pace (Are they fats and rough? Slow and sensual? etc.)

Depending on the mood, Michael can be slow and sensual, or fast and rough - but it is always a passionate exchange fuelled by your love and perpetual desire for one another. Sex with Michael is much more than a mere carnal act, but rather a complete fusion of your bodies and souls into one.

Q = Quickie (Their opinions on quickies rather than proper sex, how often, etc.)

He’s a busy man with a tight schedule, so once in a while you two simply have to make do with a quick hard fuck.

R = Risk (Are they game to experiment, do they take risks, etc.)

He’s game for experimenting within reason. As long as none of you ends up being hurt or humiliated, he’s willing to give it a try.

S = Stamina (How many rounds can they go for, how long do they last…)

Depending on how tired or well-rested he is, it spans from quickies to extended lovemaking sessions that last all night long. He rarely cums before you do, and usually that happens if you teased him too much beforehand. Even then, he will either recover and fuck you until reach orgasm as well, or he will use his mouth and fingers on you until you are satisfied. 

T = Toy (Do they own toys? Do they use them? On a partner or themselves?)

He’s not opposed to toys, but he’s an old school kinda guy, so you would pretty much have to talk him into incorporating them into your guys’ sex life. Be warned, though, he will most probably use them to edge you until you are practically crying for release.

U = Unfair (how much they like to tease)

It depends on his mood, really. Sometimes he likes to give you exactly what you want, when you want it - and sometimes he can be a complete and utter tease, to the point he drives it drives you completely nuts.

V = Volume (How loud they are, what sounds they make)

He’s relatively quiet, with a couple of low grunts and moans sprinkled in-between laboured breaths. However, he usually cums with a long groan he usually muffles against your neck or shoulder.

W = Wild Card (Get a random headcanon for the character of your choice) X = X-Ray (Let’s see what’s going on in those pants, picture or words)

Well, you know what they say: Tiny man, huge “ego “. And Michael is the perect embodiment of this phrase. The boy is hung and he knows how to it to bring you maximum satisfaction, 100% guaranteed, no returns. 

Y = Yearning (How high is their sex drive?)

His sex drive is not actually that high, but he very rarely refuses you if you initiate it. Once you get him going, however, he is relentless.

Z = ZZZ (… how quickly they fall asleep afterwards)

Usually he stays up a little longer to admire your slumbering form, as it has a calming effect on him. He will eventually fall asleep with you in his arms, once that brilliant and busy mind of his runs out of fuel.

[ryogrid]

Python Implementation of FX System Trading Agent with Deep Reinforcement Learning (DQN)

Hi, I'm ryo_grid a.k.a. Ryo Kanbayashi.

I implemented a trading agent (with decent performance) with deep reinforcement learning (DQN) to simulate automated forex trading (FX) as an practice for learning how to apply deep reinforcement learning to time series data.

This work have done with time of 20% rule system at my place of work partially: Ory Laboratory inc .

In this article, I introduce details about my implementation and some ideas about deep reinforcement learned trade agent. My implementation is wrote with Python and uses Keras (TensorFlow) for implementing deep learning based model.

Implementation

Papers referenced (hereinafter called to "the paper")

"Deep Reinforcement Learning for Trading", Zihao Zhang & Stefan Zohren & Stephen Roberts, 2019 . papers 1911.10107, arXiv.org.

State

Current price (Close)

Past returns

The difference between the price at time t and the price at time t, which corresponds to the episode

The method proposed in the paper is not intended for FX only. So, as an example, if we consider the case of stocks, I thought that we can interpret the price change as a return

4 features. 1 year, 1 month, 2 months, 3 months

These are normalized by the square root of the number of days in the period and the exponentially wighted moving starndard deviation (EMSD) in the last 60 days

(In my implementation, I replaced one day to a half-day, so a number used on calculation is double the number of days)

About EMSD in Wikipedia

For more information, please see the paper and my implementation

Moving Average Convergence Divergence (MACD)

It seems to mean the same thing as what is commonly known, but the method of calculation seems to be different, so please refer to the formula in the paper and my implementation

Relative Strength Index (RSI)

It looks past 30 days

Later features are added by me (I replaced LSTM layers which handle multiple time series data to Dense layers. So, I added several technical indicators as summary data for historical price transition)

Change ratio in price between a price of t and price of t-1

Several Technical Indicators

Moving Avarage (MA)

MA Deviation Rate

First line value of the Bollinger Bands

Seconde line value of the Bollinger Band

Percentage Price Oscilator (PPO)

Chande Momentum Oscillator (CMO)

volatility

Exponentially wighted moving starndard deviation (EMSD) in the last 60 days

Same as the value used in calculating the feature "Past returns", and same as the value used in calculating the reward

Action

Sell, Do nothing, Buy

Actions are replesented as Sell = -1, Do nothing = 0, Buy = 1

Environment buys and sells corresponding to the selected action, puts the value shown above corresponding to the action into the reward formula, and returns the value to agent

Do nothing: do nothing

Buy: close a short position and open a long position If having a long position. If not, do nothing

Sell: close a short position and open the short position If having a short position. If not, do nothing

　　　The formula for calculating reward

Reward

Calculating formula

Calculating fomula is one shown above

Each constant and variable

A{t}: the value corresponding to the action selected at time t. The value is: Sell = -1, Do nothing = 0, Buy = 1. When it used in the calculation, t indicates the time of the current episode. The formula for reward uses the actions selected at t-1 episode and t-2 episode. But A{t} which is selected at current episode is not used

μ: in the case of FX, it is equivalent to the number of currencies to be purchased. The paper says the was set to 1

σ{t}: volatility at time t. See the paper and serious implementation for the calculation

σ{tgt}: I honestly don't understand much about this. The value must be a certain percentage of the value of return for a particular period of time (not constant value) accordint to refereed papers from the paper. But performance was bad when the definition is used. So I checked the range of values that σ{t} and I fixed it to 5.0

p{t}: price at time t without considering the spread

r{t}: p{t} - p{t-1}

bp: the percentage of transaction fees, which in the case of FX is payed as spread. In the my implementation, if there was a spread of 0.003 Yen per Dollar when 1doller = 100Yen, the valuation loss is assumed to be 0.0015 Yen for one way. So I fixed bp to 0.0015

Interpretation of the reward formula in broad terms

The basic idea is to learn appropriate actions corresponding to input features from the results of transactions in the training data (I recognize that this is essentially the same as forex prediction, etc.)

However, instead of simply using trading results as a reward, the risk of holding a position represented with the recent volatility is considered and the reward is scaled with the risk value (in the paper, this may be a method that is applied to portfolio management, etc.)

In calculating the reward at a episode in time t, informatoin of A{t} selected in time t is not used. It seems strange. However certain percentage of the rewards in next episode is added (sometimes positive, sometimes negative) due to Q-learning update equation. And reward of next episode in time t+1 is mainly determined from result of a transaction corresponding to A{t}. So there is no problem.

Interpretation of specific expressions

Fomula in box brackets corresponds to the change in holding status of a long or short position for one currency bought and sold one episode ago (even if there is no change in the position). And μ corresponds to the number of currencies bought and sold

The first item in the box brackets evaluates the action one episode earlier A{t-1}. A{t-1} varies in {-1, 0, 1} and r{t} is the difference between the price at the time of the previous episode and the price at the current time. Therefore, the item takes a positive value if the price has gone up and a negative value if the price has gone down. Therefore A{t-1} * r{t} becomes positive value if you buy and the price goes up, and if you sell and the price goes down. It is a negative value if the trade was reversed one. In short, it represents whether the trade was correct or not. If the action was donot, A{t-1} values 0 and the value of the first item becomes 0

σ{tgt} / σ{t-1} is a coefficient for scaling the above value with volatility at t-1. In situations of low volatility, value will be higher and in high volatility, value will be lower. This causes agents to behave in such a way that it avoids trading in situations of high market volatility. It seems to me that the fractions with volatility appeared in the second item is works as same as one of first item.

On the second item, -bp * p{t-1} represents the one-way transaction cost per a currency and absolute value part represents wheter the transaction costs ware incurred in te previous episode. Therefore, multiplying them together gives the transaction cost per currency incurred in the previous episode

For the part of the second item that gives absolute value, without cossidering the scaling by volatility part, if the actions at time t-1 and t-2 were a "Sell" and "Buy" ("Buy" and "Sell"), the value is 2. The value = 2 means twice of one-way transaction costs were paid for closing a position and opening a position. "Do nothing" and "Sell" or "Buy" (even if the order is reversed) will result in a value of 1 and it means one-way cost was paied. However, in this case, there must be a case where the agent just kept the position and no cost should be incurred I think. But a one-way transaction cost was incurred on the fomula anyway. In the case of "Do nothing" and "Do nothing", the value is 0 and no transaction cost is incurred

Differences between my implementetion and the method proposed by the paper (some of which may be my misread)

σ{tgt} for reward scaling

If I'm understanding it right (I'm not sure...), the paper seems to have a value that changes from episode to episode (not a fixed value). But when I implemented it, the performance was reducedsignificantly. So I'm using a fixed value

In calculation the standard deviation for MACD, the period of time is set to six months instead of one year

The paper seems to use volatilities calculated over the differences in price changes (the difference between sccessive two prices) as volatility. But my implementation uses vollatilities calculated over the prices direectly

I also tried to implement using volatilities calculated over the differences in price changese, but it only reduced performance, so my implementation is not using it

All replays of a iteration are done at last of the iteration (fit method of TF is called once only per iteration)

Learning period and test period

My implementation uses 13 years of data from mid-2003 to mid-2016, whereas the paper uses 15 years of data from 2005 to 2019

The paper evaluates the study period as 5 years

I'm guessing that they had it as a study/test at 5 years/5 years, and when looking at performance in the next 5 years, they moved the study period forward 5 years and re-modeled it again

Half-day, not one-day

To increase trading opportunities and the number of trade

The reason for increasing the number of trades is we think it reduces the luck factor

Batch Normalization is included (the paper doesn't mention whether any layers are added other than the layers described)

No replaying across iterations

In my implementation, "memory" is cleared at start of each iteration and replays are random replay. Therefore the only records of the episodes used are those of the same iteration without duplication

In the paper, it is stated that the size of the memory is 5000 and replay is performed every 1000 episodes. So it seems that the contents of the memory are maintained across iterations and the replay is performed using the contents (it is not clear how the data of the episodes are selected for the replay)

The input features are normalized by scikit-learn's StandardScaler to all of them

In the paper, it is stated that only the value of the prices are normalized

In addition, because the features of the test data must be normalized in the evaluation with test data, a instance of StandardScaler is maintained that are fitted when normalizing the features of the training data and it is used to normalize the features of the test data as well

The paper refers to the volume of transactions as a fixed number of currencies. But my implementation dynamically changes number of currencies according to amount of money agent has

In the paper, it is stated that fomula for calculation of reward value should be changed when using not fixed number of currency on each transaction to achieve compounding effect. But the formula is not changed on my implementation

Procedure for creating a model for an operational scenario

How to decide which model to use

Look at the results of backtests run periodically during learning to find just the right number of iterations before overlearning occurs

If the trade performance is stable for about a year on backtesting. Except for there is significant market trande change, it is likely that the performance will be similar. If clearly market movements and price ranges are different from the training period and the period of the test data, there is no choice but to shut down the operation

If stable result is found, you do early stopping so that you can get the model at the point. Please note that in the current implementation, the total number of iterations is used as a parameter of the gleedy method. Therefore, you can't end execution early by reducing it. Therefore, for terminating execution, inserting code for it is necessary

If stable result does't show up, you can change the learning period (longer or shorter)

If you want to extend the learning period, you need to increase the number of NN units and if you want to shorten it, you need to reduce it, which was found experimentally

NN's expressive power varies with the number of units, so if it is too few, they cannot acquire even common rules in training and test data (before over-fitting begins) due to lack of representation maybe. If it is too many, early over-fitting occurs due to excessive expressive power maybe

As the performance tends to deteriorate when there is a gap between the training period and the test period, beginning of training data is moved forwards to shorten the period, and to lengthen the period, the beginning of the training period is moved backwards (if there is extra data) or the beginning of the test period is moved into the future (if you can accept to shorten the test period)

Agent trade performance simulation (USDJPY, EURJPY, GBPJPY)

Exclude about first one year for feature generation of later period and the subsequent six years (excluding non-tradeable weekends, etc) were used as training data. And remaining period (excluding non-tradeable weekends, etc) were used as test data

Since the evaluation backtest only outputs a log of option close, results graph below also shows the number of close transaction on the horizontal axis and the axis does not strictly correspond to the backtest period (periods when no closes occurred do not appear in the graph). But it almost corresponds because close operations are distributed evenly over the all period at the results below

Execution Environment

Windows 10 Home 64bit

Python 3.7.2

pip 20.0.2

pip module

tensorflow 2.1.0

scikit-learn 0.22.1

scipy 1.4.2

numpy 1.18.1

TA-Lib 0.4.17

(Though machine used for execution had GeForce GTX 1660 Super, I have disabled it at program start-up because using GPU makes learnig speed slow on the NN network scale of this model which is relatively small and I heard that fixing the seed will not be reproducible when GPU is used)

Exchange rate trends for each dataset (price of Close on dataset)

In my implementation, the price of the long position is based on p{t} + 0.0015 yen. The price of the short position is based on p{t} - 0.0015 yen. When closing, the price is the reverse of both. Taxation on trading profit is not considered.

USDJPY (2003-05-04_2016-07-09)

EURJPY (2003-08-03_2016-07-09)

GBPJPY (2003-08-03_2016-07-09)

Hyperparameters

Learning rate: 0.0001

Mini-batch size: 64

Traing data size: Approx. 3024 rates (half-day, approximately 6 years, not including non-tradeable days)

Number of positions (chunks of currency) that can be held: 1

NN structure: main layers are 80 units of Dense and 40 units of Dense

Dueling network is also implemented

NN structure

Backtesting results with test data for each currency pair

Backtesting is started with an initial asset of 1 million yen

The duration of the training data is about 6 years (excluding about first year of data sets).

Data of excluded first year and a little over a year is used to generate features of training period

The length of the testing period varies slightly from one another, but all of them are the last five years and a little ovaer a year (the period is continuous with the one of training data)

The following is just a case of early termination with a good number of iterations that have performed well. There are also a lot of results that are not good at all when training code progress is smaller and larger number of iterations than one of results below.

USDJPY

Trands when 120 iterations trained model is used

EURJPY

Trands when 50 iterations trained model is used

GBPJPY

Trands when 50 iterations trained model is used

Source Code

Repository (branch): github

Source File: agent, environment

This is the code used to evaluate the dataset in USDJPY. If you want to load a file of another currency pair, you can use the codes which are commented out on environmnt's data loading part

Trying to execute my code

If you are on Windows, you can run it as follows (standard output and error output are redirected to hoge.txt)

pip install -r requirements.py

pip install TA_Lib-0.4.17-cp37-cp37m-win_amd64.whl (required only on Windows)

python thesis_based_dqn_trade_agent.py train > hoge.txt 2>&1

My code evaluates the model every 10 iterations by backtesting at train data and test data. The bactest results are output to auto_backtest_log_{start date}.csv (log of position close only). Firstlly backtest in the period of training data runs, then backtest in the test period runs subsequently. So please use the start date and time to determine which result is you want to see

In the result csv...

Second column value is number of episode which position close occurs at (0 origin)

Ninth column value is the amount of money the agent has after closing a position

Please refer to the code to see what the other column values represent :)

A side note

Reason why LSTM is not used

It was not possible to generalize when using LSTM

I tried several methods for generalization: changing hyper parameters, changing number of units, L1 and L2 regularization, weight decay, gradient clipping, Batch Normalization, Dropout, etc. But generalization is not succeeded at all

When using LSTM, there is a case that evaluated performance at train data decreases continuously with proportion as the training iteration progresses (not just temporarily)

I wasn't sure if the side of the reinforcement learning framework's problem or the characteristics of the NNs I composed

When trying simple supervised learning, which uses the same features and NN structure to predict the up or down of an exchange rate, same trends happened

Note that the above weird phenomenon were not verified when LSTM was not used.

I'm not quite sure reason why my implementation works successfully though state transition is straight (there is no branching)

About data for training period

It is best to keep the operational period and the training period as close as possible, and not to make the training period too long

If model is trained with longer period data, the model will be able to respond to many type of market movements and range changes. Howerver when long period training is applyed to model, win rate seemed to drop

Note: Challenges I've encountered in creating several Forex system trading programs and solutions of these challenges - Qiita (Japanese)

It would seem desirable if trainging period could be kept to about 3 years, but this is the flip side of the above. And if the price range is far from training data, transactions itself did not seem to occur

Generating two models, one with short training period and the other with a long training period, and switch model to operate according to market trend might be a good idea

There is no concept of stop-loss in my agent. Therefore maybe it's better to introduce stop-loss mechanism outside of the model but it's not easy. If stop-loss mechanism is implemented simply, trade performance should be not good (from my experience)

Evaluation metric of backtest (trade simulation) result

I think that it's not enough to look at the sharpe ratio

For example, stable incresing trend and trend which has radical increasing transaction and other transactions which don't chage amount of money give almost same sharp ratio values if amounts of money are same at last of test period

Series of tweets on this matter (Japanese)

Finally

I would like to express my respect and appreciation to Zihao Zhang, Stefan Zohren and Stephen Roberts who are author of the paper I was referring to

I'd appreciate it if you point out any errors in my reading of the paper

There may be some bug on my implementation. So if you find, it would be helpful if you could regist github issue to my repository

If you have any advice, I'd appreciate your comments even if it's trivial!

Enjoy!

[jcmarchi]

DeepSeek-R1 reasoning models rival OpenAI in performance

New Post has been published on https://thedigitalinsider.com/deepseek-r1-reasoning-models-rival-openai-in-performance/

DeepSeek-R1 reasoning models rival OpenAI in performance

.pp-multiple-authors-boxes-wrapper display:none; img width:100%;

DeepSeek has unveiled its first-generation DeepSeek-R1 and DeepSeek-R1-Zero models that are designed to tackle complex reasoning tasks.

DeepSeek-R1-Zero is trained solely through large-scale reinforcement learning (RL) without relying on supervised fine-tuning (SFT) as a preliminary step. According to DeepSeek, this approach has led to the natural emergence of “numerous powerful and interesting reasoning behaviours,” including self-verification, reflection, and the generation of extensive chains of thought (CoT).

“Notably, [DeepSeek-R1-Zero] is the first open research to validate that reasoning capabilities of LLMs can be incentivised purely through RL, without the need for SFT,” DeepSeek researchers explained. This milestone not only underscores the model’s innovative foundations but also paves the way for RL-focused advancements in reasoning AI.

However, DeepSeek-R1-Zero’s capabilities come with certain limitations. Key challenges include “endless repetition, poor readability, and language mixing,” which could pose significant hurdles in real-world applications. To address these shortcomings, DeepSeek developed its flagship model: DeepSeek-R1.

Introducing DeepSeek-R1

DeepSeek-R1 builds upon its predecessor by incorporating cold-start data prior to RL training. This additional pre-training step enhances the model’s reasoning capabilities and resolves many of the limitations noted in DeepSeek-R1-Zero.

Notably, DeepSeek-R1 achieves performance comparable to OpenAI’s much-lauded o1 system across mathematics, coding, and general reasoning tasks, cementing its place as a leading competitor.

DeepSeek has chosen to open-source both DeepSeek-R1-Zero and DeepSeek-R1 along with six smaller distilled models. Among these, DeepSeek-R1-Distill-Qwen-32B has demonstrated exceptional results—even outperforming OpenAI’s o1-mini across multiple benchmarks.

MATH-500 (Pass@1): DeepSeek-R1 achieved 97.3%, eclipsing OpenAI (96.4%) and other key competitors.  

LiveCodeBench (Pass@1-COT): The distilled version DeepSeek-R1-Distill-Qwen-32B scored 57.2%, a standout performance among smaller models.  

AIME 2024 (Pass@1): DeepSeek-R1 achieved 79.8%, setting an impressive standard in mathematical problem-solving.

A pipeline to benefit the wider industry

DeepSeek has shared insights into its rigorous pipeline for reasoning model development, which integrates a combination of supervised fine-tuning and reinforcement learning.

According to the company, the process involves two SFT stages to establish the foundational reasoning and non-reasoning abilities, as well as two RL stages tailored for discovering advanced reasoning patterns and aligning these capabilities with human preferences.

“We believe the pipeline will benefit the industry by creating better models,” DeepSeek remarked, alluding to the potential of their methodology to inspire future advancements across the AI sector.

One standout achievement of their RL-focused approach is the ability of DeepSeek-R1-Zero to execute intricate reasoning patterns without prior human instruction—a first for the open-source AI research community.

Importance of distillation

DeepSeek researchers also highlighted the importance of distillation—the process of transferring reasoning abilities from larger models to smaller, more efficient ones, a strategy that has unlocked performance gains even for smaller configurations.

Smaller distilled iterations of DeepSeek-R1 – such as the 1.5B, 7B, and 14B versions – were able to hold their own in niche applications. The distilled models can outperform results achieved via RL training on models of comparable sizes.

🔥 Bonus: Open-Source Distilled Models!

🔬 Distilled from DeepSeek-R1, 6 small models fully open-sourced 📏 32B & 70B models on par with OpenAI-o1-mini 🤝 Empowering the open-source community

🌍 Pushing the boundaries of **open AI**!

🐋 2/n pic.twitter.com/tfXLM2xtZZ

— DeepSeek (@deepseek_ai) January 20, 2025

For researchers, these distilled models are available in configurations spanning from 1.5 billion to 70 billion parameters, supporting Qwen2.5 and Llama3 architectures. This flexibility empowers versatile usage across a wide range of tasks, from coding to natural language understanding.

DeepSeek has adopted the MIT License for its repository and weights, extending permissions for commercial use and downstream modifications. Derivative works, such as using DeepSeek-R1 to train other large language models (LLMs), are permitted. However, users of specific distilled models should ensure compliance with the licences of the original base models, such as Apache 2.0 and Llama3 licences.

(Photo by Prateek Katyal)

See also: Microsoft advances materials discovery with MatterGen

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is co-located with other leading events including Intelligent Automation Conference, BlockX, Digital Transformation Week, and Cyber Security & Cloud Expo.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

Tags: ai, artificial intelligence, benchmark, comparison, deepseek, deepseek-r1, large language models, llm, models, reasoning, reasoning models, reinforcement learning, test

[tjudelawnakedr-blog1]

How Guitar Can Save You Time, Stress, and Money.

In the nineteenth century the guitar’s entire body also underwent adjustments that resulted in improved sonority. It became broader and shallower, with an incredibly thin soundboard. Internally, the transverse bars reinforcing the soundboard were being changed by radial bars that fanned out underneath the sound gap.

The introduction on the double-coil humbucker within the mid-fifties solved this issue from the use of two coils, considered one of which can be wired in opposite polarity to cancel or "buck" stray fields. Some are spring-loaded and have a "whammy bar", a detachable arm that lets the player modulate the pitch by switching The strain over the strings. The whammy bar is usually also called a "tremolo bar". (The outcome of speedily altering pitch is properly referred to as "vibrato". See Tremolo for even further dialogue of this expression.) Some bridges also permit for alternate tunings on the contact of the button. Your recordings and also your custom made chords are stored as buttons which can be dragged close to to change situation. For 4 strings, the 5th fret on one particular string is similar open up-Be aware as the next string; as an example, a fifth-fret Notice on the sixth string is the same Be aware as the open up fifth string. To pick a guitar Notice, click on the corresponding string and fret. A little white circle will surface indicating the chosen Be aware. learn how to play bass guitar scales The violin-kind pegbox was changed about 1600 by a flat, a bit reflexed head with rear tuning pegs; inside the nineteenth century, steel screws were substituted for the tuning pegs. The early tied-on intestine frets had been changed by crafted-on ivory or steel frets within the 18th century. Changing the truss rod impacts the intonation of the guitar in addition to the peak of your strings from the fingerboard, called the motion. Some truss rod systems, known as double motion truss units, tighten both means, pushing the neck both of those ahead and backward (conventional truss rods can only launch to a degree further than which the neck is not compressed and pulled backward). The artist and luthier Irving Sloane pointed out, in his e-book Steel-String Guitar Building, that truss rods are meant primarily to cure concave bowing with the neck, but are unable to appropriate a neck with "back again bow" or one that is now twisted. The guitar is usually a fretted musical instrument that sometimes has six strings.[one] It is often performed with both equally hands by strumming or plucking the strings with possibly a guitar choose or perhaps the fingers/fingernails of one hand, when concurrently fretting (urgent the strings towards the frets) While using the fingers of another hand. Early amplified guitars employed a hollow human body, but solid wood guitars started to dominate in the course of the sixties and 1970s, as They are really considerably less susceptible to undesirable acoustic comments "howls". Just like acoustic guitars, There are a variety of sorts of electric guitars, like hollowbody guitars, archtop guitars (Employed in jazz guitar, blues and rockabilly) and good-body guitars, which can be commonly used in rock songs. In acoustic guitars, string vibration is transmitted through the bridge and saddle to your body through seem board. The seem board is usually made of tone woods like spruce or cedar. Timbers for tone woods are chosen for equally Guitar toughness and talent to transfer mechanical Electrical power in the strings into the air in the guitar entire body. A further class of alternative tunings are known as fall tunings, as the tuning drops down the bottom string. Dropping down the bottom string an entire tone results in the "fall-D" (or "dropped D") tuning. Its open up-string notes DADGBE (from lower to high) allow for for your deep bass D Notice, which can be Utilized in keys including D major, d minimal and G major. Electric powered guitars and bass guitars have to be used using a guitar amplifier and loudspeaker or simply a bass amplifier and speaker, respectively, to be able to make plenty of seem to become listened to with the performer and viewers. Electrical guitars and bass guitars nearly always use magnetic pickups, which create An electrical sign once the musician plucks, strums or otherwise plays the instrument. The amplifier and speaker bolster this sign utilizing a ability amplifier along with a loudspeaker. Acoustic guitars which might be Outfitted which has a piezoelectric pickup or microphone can also be plugged into an instrument amplifier, acoustic guitar amp or PA technique to generate them louder. Many different tunings could possibly be made use of. The most typical tuning, often called "Conventional Tuning", has the strings tuned from a low E, to your large E, traversing a two octave vary—EADGBE. When all strings are played open up the resulting chord is really an Em7/add11.

[nonbinaryparent]

https://amp.mindbodygreen.com/articles/one-habit-for-raising-children-who-actively-question-gender-stereotypes

The One Habit To Teach To Raise Kids Who Actively Question Gender

By Bobbi Wegner, Psy.D.

June 10, 2019

Last night I walked in late from my clinical practice and grabbed a quick dinner before bed. My kids were amped up and excited, and my son (Tyler, age 8) and daughter (Eve, age 5) played on the couch. They are like little puppies, often jumping all over each other and constantly touching, talking, and engaging, for better or worse. I mostly try to quiet the constant buzz of noise that too easily seeps into my brain, but last night I caught wind of a statement and then a question directed at me.

"Eve, you are tomboy. Mom, Eve is a tomboy, right?"

Eve continued pouncing on her brother unaware of the implication, but it gave me pause. I listened, thought, and noticed my own internal reaction.

By all outside accounts, I guess Eve is a "tomboy." She is tough as nails, rides dirt bikes, wrestles (and sometimes dominates) her big brothers, and often takes her T-shirt off when she’s hot.

I am proud of her. I was a "tomboy" too. What I wanted to say in that moment was, "Yes, Eve is a tomboy, and she does whatever the f*ck she wants." My friends joke that she is my feral child, fiercely independent. If I were to disappear for a week or two, it would be Eve who figured out dinner for everyone. She would probably ride her dirt bike to the local coffee shop and negotiate an IOU contract with a confused yet impressed barista.

The thing is, she is not a tomboy. Or a boy at all. She is herself. She is a girl. She likes dirt bikes, wrestling, winning, and can weather physical injury better than a cage fighter. She also loves puppies and playing house, and her most favorite thing is cooking with me. We share a deep love for beautiful napkins, dinner parties, fancy flowery perfume, and cool heels. She is Eve. She is cool. And she is all girl.

I push them to ask themselves: Why do we say what we say? How do we know that to be true? Where did that come from? Where did we learn that? What are we actually trying to say?

I know this personality makeup well as we share passions that are often paradoxical and opposing to the outside world. I am a woman with varied interests, and I feel all woman. Eve is a girl with varied interests, and as far as I can see now, she is all girl. It is that simple.

Although in my own inner dialogue, I want to say, "Yeah Ty, Eve is a tomboy, so watch out," that would undermine everything about those qualities being feminine, too. I would be gendering behaviors that aren't inherently gendered. My automatic response is not appropriate for the current culture anymore. And the responsibility falls on me to notice my automatic thought, process it, and say something different.

Although it might seem that I am advocating for gender equality for my daughter, this is a direct message to my son, too. The message I want all my children to hear loud and clear is that you are you. There is no expectation from me around how you should behave based on your gender. Of course, there is an expectation to be kind, polite, welcoming, hardworking, and courageous, to name a few, but those are expectations I have across the board for all of my children.

Teaching the art of questioning.

Gender stereotyping is reductive and limits the horizons of possibility, and denying expression is only harmful to our kids' health and well-being in the long run. One of my parenting priorities is to offer as much opportunity as possible so my kids have the freedom to choose their path themselves. I want to build self-efficacy. Ultimately, my hope is that they lead an empowered and self-led life. Training starts now, not in adulthood. We need to be helping our kids build a sense of self from the inside out.

So, in the moment, I teach Ty that tomboy is a dated word that doesn't apply anymore. But what I am really saying is it is OK to be you—in whatever form that is. I constantly challenge them to think critically about the language they use. So much is absorbed from school and the outside community, and the best defense I have is to teach intellectual and emotional curiosity.

I push them to ask themselves: Why do we say what we say? How do we know that to be true? Where did that come from? Where did we learn that? What are we actually trying to say?

For example, any time they say, "that is a girl toy or a boy toy," I ask them to think about what makes it so. Inevitably, they can see the flaw in their assumption; they learn to self-reflect, wrestle with meaning, and what I love the most—it removes me from the often annoying power position of teacher saying yay or nay. I can join them in an intellectual exercise rather than impose a way to behave.  

Much of the meaning we create as a culture is so nuanced and implicit that we all participate in perpetuating ideas that we do not even buy into it. The first step in teaching kids to be open is to notice our own bias and challenge ourselves. We need to ask ourselves the same questions we ask our children.

Why do we think this? Where did it come from? What are we actually trying to say? It not only combats gender inequality and toxic masculinity, but it pushes kids to be more self-aware and tighten the language they use, not relying on automatic (what I call) "lazy language." Say what you actually intend to say.

Article continues below

Unlock The Power Of Food To Heal Your Body, Prevent Disease & Achieve Optimal HealthSign up now and access your FREE exclusive Functional Nutrition Webinar to get you on the path to optimal healthYou'll also get daily wellness inspiration right in your inbox!

Taking small steps.

Listen, some parents will be far more comfortable and at ease dispelling gender than others, and that mostly has to do with our own progress toward questioning and deconstructing gender, a process that many adults are still going through themselves. That's OK. Some parents are bravely raising "theybies" and "gender-creative kids," for example, and that's amazing; meanwhile, for many other parents, that's just outside the scope of what they're capable of right now, no matter how much they want to support gender diversity.

But all parents can teach their kids the art of questioning gender. If you want to raise kids who feel free to explore their identity with complete freedom, it starts with raising kids with access to the tools of curiosity and open-mindedness. Raise kids who are open to following their interests, willing to explore, and committed to noticing the meaning embedded in our culture. Although those may seem like a tall order for such young kids, these are traits that are easiest to internalize the younger they are.

Here are a few simple habits to consider introducing in your household and parenting style to encourage a culture of questioning gender:

Notice any gendered language (i.e., anytime someone says someone did something because "they are a boy or a girl"). Notice it, name it, and encourage your kids to get curious about what that means.

Offer all options to all children. Boys are welcome to play house, and girls are welcome to play in the dirt. If the children naturally segregate, split time in both activities.

Create mixed-gender groups at home, with friends, and at school. Avoid the desire to tease your kids about "having crushes" on their friends of other genders.

Especially with boys, empathize and validate emotion. Positively reinforce sharing of all emotion with words. Help boys build a strong sense of identity that includes caring for and loving others.

Manage your own discomfort when your child wants to explore something outside of their culturally prescribed behavior (i.e., Eve wanting to take her shirt off, or Ty wanting to sit on the toilet to pee). Children are honest and will give feedback if it makes others uncomfortable. Your job is not to promote or rescue but rather to follow your child's lead.

Notice how work is divided in your home. What nonverbal messages are you sending? Be self-aware and reflective and see if your behavior matches your intentions.

Lastly, be self-compassionate. Know you will make missteps. So will your kids. Be honest. Be curious. And create an open dialogue.

[hueynomure]

Summoning Disaster - Part Two: Pandora’s Piñata

In which two of my OCs end up in @fatal-blow 's Golden Verse. Set in an unspecified point before Defenders of Earth's prologue; You can find out more about the story on his wip page.

First part

Third part

Fourth part

Here extrapolations start to wear thin and I have to invent things whole cloth. I'm open to edit things to be more in line with canon if feasible. Also, my OCs come from a more high fantasy setting, so their power sets are rather broad if compared to the Defenders.

(the title of the first part was inspired by an Imagine Dragon's song, the second by an album by Diablo Swing Orchestra)

Edit: I'm going to tweak this piece to better reflect the canon.

* * * * *

Once the guards were dealt with, Amp’s team took in the scene.

Aside from the parts that had been blown off, the walls and roof of the large room were scarred irregularly, soot and cracks forming shapes that hurt the eye if looked at for too long. The bare concrete floor was more evenly charred, with clear spots where equipment – and people, judging by the lingering smell – had been incinerated, but the snaking lines of a very articulated summoning circle had been etched on it with mechanical precision. And at the very center of the room, a large grey sheet embroidered with small glimmering stones and multicolored threads, buzzing with power, was covering something hemispherical and about ten feet wide.

Amp felt Norman’s focus wander on the circle and the shapes on the walls. “Reaper?”

“Never seen a circle this complicated. And the walls… it’s like the explosion was trying to add to it,” he replied with a frown.

Someone cleared their voice. The team turned as one toward the center of the room.

“I had hoped for a more badass one-liner, to be honest, but points for the spooky I guess.” The husky voice coming from the hemisphere had some hiss to it, and suggested the owner was rather large.

“A literal shroud,” Reaper whispered, “makes sense.”

Amp nodded. “Odds are there’s another creature in there,” he whispered, and met Ethereous’ gaze. “Uncover and suppress them, slow as you like. Don’t display threats, everyone, but be ready for things to get messy.”

Ethereous nodded and cautiously approached the shroud. He grabbed a corner and looked back at Amp, who nodded. The shroud was thrown aside with a faint jingle. The hemisphere turned out to be a cage, its bars fixed directly in the floor, and inside it…

Amp blinked in silence, and the creature did the same to adjust their sight to the sudden light. The creature looked like… like a large, one-armed, muscular anthropomorphic dinosaur dressed up for a ren fair. Iron-grey scales, spiky head, crocodile-like eyes, long thick tail ending with an arrow-shaped bone, thick off-white tunic and leather bracer. Amp smothered his surprise to reboot his brain. The creature was alone, but kneeling off-center in the cage, as if giving space to an invisible companion.

The creature raised their snout to sniff around and grimaced in Ethereous’ direction. “Please stop that, this cage isn’t for show.” The lizard stretched its hand toward the bars – not in Ethereous’ direction, Amp noticed, they were just proving a point – and retracted it when a black flame sparked from the point where the lizard’s fingers were about to touch them. “And your magic’s stench is giving this shit a run for its money,” the lizard added gesturing at the cage.

The lizard was able to smell magic, apparently. Ethereous glanced back at Amp. Amp covertly gestured for him to back away and nodded, then noticed the lizard had followed Ethereous’ gaze and was staring at him in the eyes. Now Amp could feel the lizard, frustrated and excited and cautiously hopeful, and its hidden companion, curious and eerily calm. Red and blue. Amp slipped within the lizard to know more, finding a white-hot tangle of vicious anger kept on a short leash-

Amp reeled backwards, his head spinning. The lizard kept staring at him, tail swinging from side to side. He felt his team’s anxiety spike from far away, and blindly reached to keep it under control.

“Let’s keep our talents to each own, fellow Heartstoker.” The lizard sized up Reaper and Quantum. “I hope you didn’t come here just to kill assholes and taking turns to piss me off, I hoped for something more… I don’t know, civilized.”

Amp took a deep breath to clear his head. Another empath… they must have used that repressed fury to bludgeon him away somehow. He reached more subtly, feeling the lizard taming their own emotions. No reaction. The lizard had merely caught him off-guard, he just had to keep himself on the surface. The invisible companion showed no change. “Alright,” he sighed, “let’s be civilized. I’m Amp, who are you?” There was no reaction for the name in either creature.

“Sharaka. What’s your goal here?”

“Learning what the ‘assholes’ did in this room. Who’s your invisible friend there?”

“My name is Elphimas,” said a measured voice. “And I might be able to improve your understanding of what happened, though I do not know whether I should yet.”

“Amp, how are things around there?” Static. He felt slightly tenser than he should.

Amp covered his mouth with a hand. “Trying to figure out things. Anything wrong?”

“Not yet, but there are guys with weird armors that block sparks and light like nothing, they’re a pain in the ass.” The noise of a violent discharge, gunshots. “Bullets work fine though.”

“Roger that. Quantum, step back and watch over Static’s team.” Amp rolled his shoulders back and met Sharaka’s eyes again. “I apologize for my rudeness,” he said, voice dripping with sarcasm, “but I’m on a clock here. What do I have to do to get an explanation?”

“Letting us out of here would be a great start,” Sharaka replied.

“Cracking the crystal on the top of the cage should suffice. I suggest a ranged attack,” supplied Elphimas. Amp felt vague concern. “As a gesture of good faith, I can tell you we are not the beings the ritual was supposed to summon. Luckily for this plane, I might add.”

Amp took a few steps forward to get a better view of the crystal Elphimas was talking about. Fist-sized, with some black substance whirling within it. Would probably backfire if hit from short distance. “Two questions. One: how can I know you’re not going to attack us as soon as you’re free?”

“How can I know you’re not just going to put us in another fucking cage?” Sharaka snarled. “You look like tough guys and did nothing stupid. Two?”

“I don’t want you to leave you with the ‘assholes’. Will you play nice and come with us?” Amp tapped idly on his gun.

Sharaka sniffed the air. Hope and distrust spiked at once. Did she smell emotions too? “I have to get a few things back from them,” Sharaka objected after a long moment. “Then I’ll decide.”

“Retrieving her weapon in particular would be in both our groups’ interest,” added Elphimas.

“Why? And could you please fucking show yourself, Elphimas?”

A second kneeling figure appeared next to Sharaka. Elphimas looked human, and very… nondescript. Aside from the ren faire clothes, their skin was vaguely tan, their hair mud-brown and their dull eyes a mixture of grey, green and brown. Amp was rather sure it wasn’t their true appearance, but it was marginally better than speaking to thin air. Elphimas smoothed nonexistent creases in their clothes for a moment, hesitant, then jerked their head aside as if they had heard something. “Your teammates are probably going to give you the reason momentarily,” they said with another hint of concern.

The earth shook with a low rumble, and right on cue Static opened comms. “Now would be the right time for the cavalry, Amp! They’re so mad they’re dropping the building on us!” Amp felt the faint shadow of worry at the bottom of Static’s laughter. “More importantly, I’ve been cut off from the others.” Gunshots, crackling of electricity, silence. “Wouldn’t want to outshine them too badly, know what I mean?”

Sharaka’s tail slammed on the ground. She was literally fuming - Amp could see air trembling around her. “Bastards, that’s my weapon! I’ll kill them all!” She snarled and slammed her tail down again. “Get me out of here and you won’t find the ashes of those fuckers!”

“Sharaka’s weapon can destroy almost anything on contact if imbued with enough energy,” Elphimas explained.

Amp took his gun out as the image of a cornered Static filled him with cold fury. Let Sharaka rampage, he could handle her if push came to shove and worst case scenario they could leave her body behind. The crystal exploded in a column of black fire when he shot it. He opened comms with the rest of the Defenders. “Reaper and Ethereous will threaten Lab 2 to take pressure off diversion team. Windfall and Anyform, don’t let reinforcements come near Lab 1. Static, Adamantine, broadcast your locations, I’m coming. Link, prepare Lab 3 for demolition. Quantum, help Link and blind Lab 1.” A noise of straining metal made Amp turn back to the cage.

Sharaka’s first kick had bent the steel bars. The second opened a hole in the cage, which she casually widened with her hand until she could squeeze through. “Elphimas. Pack.” she hissed, and the next moment she took off at dead run, shattering the door as she sped through it.

Amp opened comms again. “If you see a humanoid lizard, get out of her way. Repeat: humanoid lizard, get out of the way. Elphimas, you’re coming with m-”

But the “human” had already disappeared. Amp looked for Elphimas’ mind nearby, with little hope. He allowed himself three precious seconds of anger before sharpening himself again.

[racingtoaredlight]

Random Thoughts on Bass

I’ve gone wayyy down a bass wormhole, so you’ll all have to deal.

Technically, bass came extremely easy considering all the years I played guitar and had already known how to build intermediately complex basslines.  However, I don’t know if it’s DNA or memory residue, but I’ve found myself still drilling down technique quite a bit in practice.

There is a law of diminishing returns for a reason, though.

***

If you watched the above video (and trust me, I wouldn’t blame you if you didn’t), you’d see what seemed to be a pretty solid case and demonstration of a technique that could be incredibly beneficial to a bassist playing more complex music.

Years ago, I wrote about the idea of musical geometry and how some of the more authoritarian composers like Bach, Beethoven and Wagner would build these musical architectures out of harmonies and the piece’s form.  All the different elements come together in synchronized harmony to create a piece of music.  Here’s a simplified visualization.

Theoretically, this idea is evident in how ideas are formed, established, reinforced and how they exist in harmony when performed.  Thinking of Bach’s music in terms of fractals is especially correlated as it’s perfectly constructed with very little ornamentation or “unimportant” notes.

And when I’ve talked about acoustics, I’ve talked about how lower pitched notes have “bigger” sound waves.  Music is a physical being...it is physical sound waves moving through the air.  When you look at sound waves, volume is determined by amplitued (how high the sound waves crest) while pitch is determined by wavelength (distance between high points).  It’s why orchestras can have two sections of 18 violins, and be totally cool with 8 bassists.

What I’m getting at with this is this...

Playing bass like in the video at the top is such a distorted, disfigured, tortured fashion of what a bass player should be, I feel bad for the guy.  The amount of time and energy and intellect and practice and frustration it took to learn how to play like that is hard to sum up.  Even for a prodigy like Matthew Garrison.

Which leads to the most important question I believe he answered incorrectly...WHY?

***

He believes, and I don’t think he’s being falsely modest, that he wasn’t born with good enough physical ability to play bass at the speed the other musicians in Joe Zawinul’s band (keys, sax and other horns, drums) could play with.  By adopting this 4-fingered right hand technique, and playing a bass on a stand we’ll talk about in a second, he could play these blazingly fast sax and keyboard lines.

...if they were looking for someone to play those blazingly fast sax and keyboard lines, they wouldn’t have hired a bassist.

Matthew Garrison kept his job because, like I said, he’s a prodigy bassist.  Intellectually, he’s on a level that is really, really hard to explain.  Like, remember how I explained theoretical fractals?  He takes complexity and makes fractals with more complexity.

Here he is playing Coltrane’s “Giant Steps”...one of the most difficult set of harmonic changes that exist in Western music, while still being musical...in 7/8 time, and accompanying himself soling with these crazy chords.

***

Yall in the comments have criticized me for being overly technical or not embracing the beauty of simplicity, and trust me...I totally get it.  I watch a video like this and totally understand where you’re coming from.

I’m completely unmoved by this.  I can appreciate the technique and mastery of theory in real time, but I never want to listen to this again.

It’s hard to criticize Garrison because he’s so advanced.  He mentioned Jaco though...Jaco played in Zawinul’s band (that’s putting it generously...the truth is closer to Jaco is the guy Zawinul hired Garrison to replicate), and he didn’t need this contorted 4-finger technique, nor did he need some douchey guitar stand instead of a strap.

He might have been playing alien jazz shit, but Jaco was down-deep, a redneck from Florida playing Chitlins Circuit funk.  He always had soul and he always had a groove.  Jaco took bass to new places, but he was always a bassist and filled that role first and foremost.

This music has no groove.  Garrison rarely has any groove.  He has these flashy solos, these advanced concepts, these intricately arranged parts...but not a single bit of it is memorable.  And all I can help but think is if he wanted to solo this bad, and spent this much time on it...why is he playing bass to begin with?

***

Take a look at his bass.  It’s a no-doubt work of art in terms of craftsmanship, and I personally love Fodera basses, so I’m not going to call them garbage but...

Remember how I talked about the beauty of simplicity when talking about the Fender Precision Bass?  One pickup, one volume and one tone control...and that’s it.  And while I love Fodera’s and they’re played by the best bassists on earth, something seems contradictory about them...

They’re made of these insanely beautiful tone woods, which they painstakingly go through the work of explaining the sonic differences of...but then they slap active pickups, a pre-amp, and so many electronics that it needs TWO 9v batteries to power.

Why use incredible tone woods if you’re going to cover it all up with solid-state electric stuff?

That big block in the middle of his two pickups?  It’s called a “ramp.”  What it allows a bassist to do is to minimize the amount of effort expended to pluck the string...thereby reducing the time your finger is on the string and allowing you to play faster.

Again, you have these incredible tone woods already covered up by electronics, and now you install something that reduces the vibrations of the strings even more?  Like, the farther a signal has to travel, the more filters and preamps it goes through, the more its original signal gets degraded...look at the comparisons of a P-Bass’ electronics to the ones you’ll see in a Fodera.

Here’s the Fodera.

Here’s the P-Bass...

I’m not saying Fodera’s are garbage (farrrrr from that)...just that there’s contradictory things going on with their design, things that are meant to improve aspects of a bass’ performance that aren’t an important role that the instrument should serve.  A good defensive first baseman is a nice luxury, but if you hired that player to hit 40 HR’s and smack the ball all over the field but they’re Doug Minkevitch slapping singles .250 the time instead, is that really a good thing?

In a vacuum having extremely developed niche skills are fantastic.  But in a team setting, if there’s not that balance between all the different sets of skills, it’s going to be hard to be successful.  Same with a musical team where the fulcrum of your lineup (the bass) is busy trying to show off defensive skills like a shortstop who was built to do such things.

***

Which leads me to my deep dive into bass culture.

It’s just a whole damn lot of missing the fucking point.  Bass is the ultimate BIG PICTURE instrument...even when you look at the bass notes’ sound waves, you see that it’s about the BIG PICTURE.  What Garrison and a lot of bass culture do is focus on the tiny, minute details in the fractals underneath a microscope on this atomic, granular level...and in the process...completely miss the fucking point of PLAYING BASS.

Who the fuck gives a shit about a bassist tapping?  Even slap...the ultimate parlor trick...is nothing more than “look at me!” wankery.  Is that bad?  I don’t know if it is or not, but the point is that every time a bassist does this flashy nonsense or steps into the spotlight to do some slappy wappy bullshit, the music loses a fundamental element.

The bass is a foundational instrument, theoretically speaking.  It defines the key, it sets the groove or rhythm, and it serves as the musical liason between the composition, the beat and the melody.  Jaco served this role in a visible fashion, but he did not abandon the crucial responsibility a bassist serves in a band.

***

Listen to the first 30 seconds of the Wayne Shorter piece and then listen to as much as you can stomach of the bass duet version.

The whole purpose of this post was to get old man lawn angry at the idea of missing the fucking point.  You have these two bassists with more ability than ever is necessary jamming out to one of the absolutely iconic bass riffs in jazz and you never would have known it.

They’re even playing the same notes as the groove.  But there isn’t any.  There’s nothing there.  This music is soulless and vapid.  You have this faux artistic black and white camera nonsense that’s just as unnecessary as these two bassists creating a musical mudpit because fast soloing simply doesn’t work as well in lower registers.

Think about it...would you try to beat a Ferrari off the line at a red light in a Jeep Wrangler?  Then why waste time souping your Wrangler up so that it will?

Meanwhile bassists Paul McCartney is a millionaire while Sting and Gene Simmons are both worth over $300 million.  There are a bunch of bands who could use bassists as talented as these two guys to make music, and yet they’d rather try to teach a dolphin how to walk on a leash.

[jbaquerot]

Roadmap

Part 1: Why Machine Learning Matters. The big picture of artificial intelligence and machine learning — past, present, and future.

Part 2.1: Supervised Learning. Learning with an answer key. Introducing linear regression, loss functions, overfitting, and gradient descent.

Part 2.2: Supervised Learning II. Two methods of classification: logistic regression and SVMs.

Part 2.3: Supervised Learning III. Non-parametric learners: k-nearest neighbors, decision trees, random forests. Introducing cross-validation, hyperparameter tuning, and ensemble models.

Part 3: Unsupervised Learning. Clustering: k-means, hierarchical. Dimensionality reduction: principal components analysis (PCA), singular value decomposition (SVD).

Part 4: Neural Networks & Deep Learning. Why, where, and how deep learning works. Drawing inspiration from the brain. Convolutional neural networks (CNNs), recurrent neural networks (RNNs). Real-world applications.

Part 5: Reinforcement Learning. Exploration and exploitation. Markov decision processes. Q-learning, policy learning, and deep reinforcement learning. The value learning problem.

Appendix: The Best Machine Learning Resources. A curated list of resources for creating your machine learning curriculum.

[ai-landing]

2018/1/27 Feed Summary

The Morning Paper

One Model to Learn Them All: In this post, the author summarize a paper which introduces a MultiModel general deep learning framework which will train 8 tasks at the same time (Image recognition, Image caption generation, Speech recognition, Parsing, German/French to English Translation and their reverse English to German/French translation). MultiModel consists an encoder / decoder architecture which would share a common learning unit. It also consists of a mixture-of-expert layer to dispatch the learning efforts. The article referred in this post shows that training such mixed model doesn't pose any performance degradation problem and sometimes help the task with less data available (parsing). This might imply a much larger scale or cross-domain transfer or multi-tasking learning experiment could be done in the future. This article also points out even though the computational building blocks need to be present for some specific domain (convolution neural network for image and attention / mixture-of-expert for language model), their presence does not interfere the cability of learning other tasks in different domains.

(RW: From the first glimpse of this short summary, the article referred in this post doesn't show any convincing result that cross-domain training does work by showing how much different these domains are. It just iterate similar conclusion from past experiments. Does Image caption take on the role to bridge image and language domain? How about choosing tasks randomly to break the MultiModel in order to know the degree of correlaiton shared among models? Would a sub-model which is trained insufficiently take the whole unified model down? I think those questions might shed some light about cross-domain learning)

KDNuggets

Kogentix Automated Machine Learning Platform: Another MLaaS targets bussiness data (pipeline in the figure below) and features the only one platform running Spark natively.

Data Enginner Introduction Part 1: In the following figure borrowed from The AI Heirarchy of Needs, AirFlow (monitoring tool used by Airbnb) locates at second layer of AI Needs Pyramid

The Democratization of Artificial Intelligence and Deep Learning: Free e-Book give-away. The Democratization of Artificial Intelligence is an idea to make Artificial Intelligence applicaiton is accessible to everyone.

Data Science Job Market Trends: automatino, data enpowerment, mass cleanup, ethnics & influence and blockchain app

O'Reilly Media / AI

tensoflow + mobile device: introduce Tensoflow Lite

using Apache MXNet for anomalty detection: Tutorial for using MXNet. Tranditional methods used to detect anomalty including: Kalm filter, KNN, K-Means and autoencoder with DL. Using IoT time-series data for demonstration. The author will train a encoder-decoder and detect anomalty as any data point outside 3rd standard deviation.

LSTM introduction with Tensorflow: using LSTM to classify Stock Tweets

2018 trends in AI O'Reilly version: And Include

Bayesian methods into Deep Learning and optimize training through neuro-evoluation on gradient-based deep learning.

Low cost hardware to improve computation efficiency.

Fast evolve AI tools including simulators (including reinforcement learning to automate deep learning training such as AutoML), AI develop toolbox handling more complicated / multimodal inputs and finally tools that not for data scientist or AI enginner for use such as friendly UI / UX etc or Intelligent wearables alike

Replace low-skilled tasks with automation

Other ethnics or issues about AI application

Convolution NN for language modeling: tutorial using 1D kernel and Tensorflow > Written with StackEdit.