#Assistant Computer Officer | Explore Tumblr posts and blogs

jcmarchi · 3 months ago

Text

How to 8‑bit quantize large models using bits and bytes

New Post has been published on https://thedigitalinsider.com/how-to-8%e2%80%91bit-quantize-large-models-using-bits-and-bytes/

How to 8‑bit quantize large models using bits and bytes

Deep learning is consistently changing so many fields, from NLP (natural language processing) to computer vision.

However, as these models continue to grow in size and complexity, the demands on the hardware required for memory and compute continue to skyrocket. In light of this, there are promising strategies to overcome these challenges, one of which is quantization. This lowers the precision of numbers used in the model without a noticeable loss in performance.

In this article, I will dive into the theoretical processes underlying this strategy and show the practical implementation of 8‑bit quantization within a large parameter model, in this case, we will be using the IBM Granite model and BitsAndBytes for quantization.

Introduction

The quick growth of deep learning has resulted in an arms race of models boasting billions of parameters, which, in most cases, achieve stellar performance but require enormous computational resources.

As engineers and researchers look for methods to make these large models more efficient, quantization has shown to be an incredibly effective solution. By lowering the bit width of number representations from 32‑bit floating point to x‑bit integers, quantization decreases the overall model size, speeds up inference, and cuts energy consumption, all while keeping a high accuracy in the output.

I will explore the concepts and techniques behind 8‑bit quantization in this article. I will explain the approach’s benefits, outline the theory behind it, and walk you through the process step by step.

I will then show you a practical application: quantizing the IBM Granite model using BitsAndBytes.

Understanding quantization

At its core, quantization is the process of mapping input values from a quite large set (usually continuous and high-precision) to a much smaller and more discrete set, which has lower precision. Deep learning typically involves converting 32‑bit floating‑point numbers to x‑bit integer alternatives.

The result is a massive reduction in memory usage and computation time.

Benefits of quantization

Lower memory footprint: Lower precision means that each parameter requires much less memory.

Increased speed: Integer math is generally much faster than floating‑point operations (FlOps), especially on hardware optimized for low‑bit computations.

Energy efficiency: Lower precision computations consume far less power, making them ideal for mobile and edge devices.

Types of quantization

Uniform quantization: This method maps a range of floating‑point values uniformly to integer values.

Non‑uniform quantization: Uses a more complicated mapping based on the distribution of the weights or activations of the network.

Symmetric vs. asymmetric quantization:

Symmetric: Uses the same scale and zero‑point for positive and negative values.

Asymmetric: Allows different scales and zero‑points, which is useful for distributions that are not centered around zero.

AI assistants: Only as smart as your knowledge base

AI assistants need real-time, seamless connections to your company’s databases, documents, and internal communication tools to realize their full potential.

Why 8‑bit quantization?

8‑bit quantization is when each weight or activation in the model is fully represented with 8 bits, thus offering us 256 discrete values.

This approach helps maintain compression and precision by enabling:

Memory savings: Lowering the uint from 32 bits to 8 bits per parameter can cut the memory footprint by up to 75%.

Speed gains: Many hardware accelerators and CPUs are fully optimized for 8‑bit arithmetic, which massively improves inference times.

Minimal accuracy loss: With careful calibration and potentially fine‑tuning, the degradation in performance with 8-bit quantization is often minimal.

Deployment on edge devices: The reduced model size and faster computations make 8‑bit quantized models perfect for devices with limited computational resources.

Theoretical underpinnings of quantization

Quantization is thoroughly rooted in signal processing and numerical analysis. The objective here is to reduce precision whilst also controlling the quantization error, the difference between the original value and its quantized version.

Quantization error

Scale and zero‑point

A linear mapping is normally used to perform quantization:

Scale (S): Sets the step size between our quantized values.

Zero‑point (Z): The integer value assigned to the real number zero.

The process normally involves a calibration phase to determine the optimal scale and zero‑point values. This is then followed by the actual quantization of weights and activations.

Quantization Aware Training (QAT) vs. Post‑Training Quantization (PTQ)

Quantization Aware Training (QAT): This integrates a simulated quantization into the training process, allowing the model to adapt its weights to quantization noise.

Post‑Training Quantization (PTQ): Applies quantization to a pre‑trained model using calibration data. PTQ is simpler and faster to implement but it may incur a slightly larger accuracy drop compared to QAT.

Steps in 8‑bit quantization

Applying 8‑bit quantization includes some essential steps:

Preprocessing and calibration

Step 1: Investigate the Model’s Dynamic Range

Before quantization, we need to know the weights and activation ranges:

Collect Statistics: Pass a part of the dataset through the model to collect statistics (min, max, mean, standard deviation) for all the layers.

Establish Ranges: Based on these statistics, create quantization ranges, possibly clipping outliers to create a tighter range.

Step 2: Calibration

Calibration is the process of selecting the best scale and zero-point for each tensor or layer:

Min/Max Calibration: Uses the minimum and maximum that were observed.

Percentile Calibration: Uses some percentile (e.g., 99.9th percentile) to avoid outliers. Calibration must be correct since poor decisions will result in significant loss of accuracy.

Quantization Aware Training vs. Post‑Training Quantization

Quantization Aware Training (QAT):

Advantages: Greater precision as the model learns how to compensate for quantization distortion.

Cons: Involves modifying the training procedure and extra computation.

Post‑Training Quantization (PTQ):

Advantages: It’s much easier to implement because the model is already pre-trained.

Disadvantages: It can sometimes result in a greater reduction in accuracy, specifically in precision-based models.

For most big models, a small loss of accuracy from PTQ is fine, while mission-critical applications can use QAT.

LLM economics: How to avoid costly pitfalls

Avoid costly LLM pitfalls: Learn how token pricing, scaling costs, and strategic prompt engineering impact AI expenses—and how to save.

8-bit quantization applied

No matter which deep learning environment—PyTorch, TensorFlow, or ONNX—the concepts of 8‑bit quantization remain the same.

Practical considerations

Before implementing quantization, consider the following:

Hardware support

Ensure that the target hardware (CPUs, GPUs, or special accelerators like TPUs) natively supports 8‑bit operations.

Libraries

PyTorch: Gives us built-in support for QAT and PTQ through its designated quantization module.

TensorFlow Lite: Offers us utilities to transform models to an 8‑bit quantized format, especially for embedded and mobile applications.

ONNX Runtime: Supports quantized models for use across different platforms.

Model Structure: Not all the layers in the model are created equal when quantized.

Convolutional and fully connected layers will generally be fine, but some activation and normalization layers may need further special treatment.

Fine-Tuning: Fine-tuning the quantized model on a small calibration dataset can help restore any performance loss due to quantization noise.

BitsAndBytes: A specialized library for 8‑bit quantization

BitsAndBytes is an independent library that helps us further streamline the 8‑bit quantization process for very large models. Frameworks like PyTorch offer us native quantization support. However, BitsAndBytes provides additional optimizations designed to convert 32‑bit floating point weights into 8‑bit integers.

With a simple config flag (e.g., load_in_8bit=True), it enables significant reductions in memory usage and speeds up inference without requiring massive code modifications.

Model structure: Not all layers are equally amenable to quantization. Convolutional and fully connected layers usually perform well under quantization, but some of the activation and normalization layers may need special treatment.

Fine‑tuning: Fine‑tuning the quantized model on a small calibration dataset can help us recover any performance loss due to quantization noise.

Integrating BitsAndBytes with your workflow

For seamless integration, BitsAndBytes can be used alongside other popular frameworks like PyTorch. When you pre-configure your model with BitsAndBytes, you simply have to specify the quantization configuration during model loading.

This tells the system to automatically convert the weights from 32‑bit integers to 8‑bit integers on the fly thus reducing the overall memory footprint by up to 75% and enhancing inference speed, which is ideal for deployment in resource-constrained environments.

For example, by setting up your model with:

quantization_config = BitsAndBytesConfig(load_in_8bit=True)

you can achieve a quick switch to 8‑bit precision. This approach not only optimizes memory usage but also maintains high performance, making it a valuable addition to your deep learning workflow.

Case study: Quantizing IBM Granite with 8‑bit using BitsAndBytes

IBM Granite is a 2‑billion parameter model designed for instruction‑following tasks. Due to its enormous size, it is possible to quantize IBM Granite to 8‑bit to reduce its memory footprint significantly with good performance.

IBM Granite quantization: Example code

The following is the code segment for configuring IBM Granite with 8‑bit quantization:

# Setup IBM Granite model using 8-bit quantization. model_name = “ibm-granite/granite-3.1-2b-instruct” quantization_config = BitsAndBytesConfig(load_in_8bit=True) model = AutoModelForCausalLM.from_pretrained( model_name, quantization_config=quantization_config, device_map=“balanced”, # Adjust as needed based on available GPU memory. torch_dtype=torch.float16 ) tokeniser = AutoTokeniser.from_pretrained(model_name)

Code breakdown

Model Selection:

The model_name variable sets up the IBM Granite model to be used for instruction execution.

Quantization Setup:

BitsAndBytesConfig(load_in_8bit=True) activates 8‑bit quantization. It is a flag that informs the model loader to quantize 32‑bit floating point to 8‑bit integer.

Model loading:

AutoModelForCausalLM.from_pretrained() loads the model using the specified configuration. The parameter device_map=”balanced” helps distribute the model across available GPUs, and torch_dtype=torch.float16 ensures that any remaining computation uses half‑precision.

Tokenizer initialization:

The tokenizer is instantiated with AutoTokeniser.from_pretrained(model_name) and guarantees the input text undergoes correct preprocessing for the quantized model.

This method not only lowers the memory usage of the model by as much as 75%, it also increases inference speed, making it particularly suitable for deployment in memory-limited settings, such as edge devices.

Gold-copy data & AI in the trade lifecycle process

Use AI to streamline the trade lifecycle, reduce manual breaks, and sync data across systems for faster, more accurate investment decisions.

Barriers and best practices

Even though 8-bit quantization is highly advantageous, it also has some challenges:

Challenges

Accuracy degradation

Some models can suffer from a loss of accuracy after quantization due to quantization noise.

Calibration difficulty

It is important to determine appropriate calibration data and techniques and may be difficult, especially for models with a broad dynamic range.

Hardware constraints

Ensure that your target deployment platform fully supports 8‑bit operation, or performance will be disappointing.

Best practices full calibration

Use a representative data set to accurately calibrate the model’s weights and activations.

Layer-by-layer analysis

Determine which layers are sensitive to quantization and evaluate the necessity to retain them at a higher precision.

Progressive evaluation

Quantization is not a one-shot fix. Repeat your strategy in turn experimenting with different calibration techniques and potentially mixing PTQ with QAT.

Use framework tools

Utilize the high-level quantization utilities integrated into frameworks such as PyTorch and TensorFlow, as these utilities are always being improved and updated.

Fine‑tuning

If possible, optimize the quantized model on a subset of your data to recover any performance loss due to quantization.

Conclusion

Quantization and 8‑bit quantization are powerful techniques for reducing the memory footprint and accelerating the inference of large models. By converting 32‑bit floating‑point values to 8‑bit integers, you can achieve significant memory savings and speedups with minimal accuracy loss.

In the current article, we discussed the theoretical foundations of quantization and expounded on the steps involved in preprocessing, calibration, and choosing between quantization-aware training and post-training quantization.

We then gave practical examples using popular frameworks, finishing with a case study involving the quantization of the IBM Granite model using BitsAndBytes.

As models in deep learning increase in size, mastering techniques like 8‑bit quantization will be needed to deploy efficient state‑of‑the‑art systems: right from the data center down to edge devices.

Regardless of whether you’re an AI researcher or a deployment engineer, understanding how to make large models optimized is a needed skill in today’s AI landscape.

The application of 8-bit quantization through tools such as BitsAndBytes allows the reduction of the computational and memory overhead of big models, such as IBM Granite, to be achieved for more scalable, efficient, and energy-consumption-friendly deployment in diverse applications and hardware platforms.

Happy quantizing, and may every bit and byte count in your models become leaner, faster, and more efficient!

Connect with like-minded AI professionals and enthusiasts at our in-person events across the globe.

Check out where we’ll be this year, and join us to discuss emerging topics with some of the world’s leading AI minds.

AI Accelerator Institute | Summit calendar

Unite with applied AI’s builders & execs. Join Generative AI Summit, Agentic AI Summit, LLMOps Summit & Chief AI Officer Summit in a city near you.

0 notes

whoevenisjavier · 2 months ago

Text

a prize i’d cheat to win

pairing: CEO harry castillo x exec. assistant f! reader

summary: you fuck your married boss during a late night at the office.

part 2 here

a/n: so… this is like… heavy cheating stuff. if that’s not your thing, then best to stop now

tags/warning: +18, mdni. harry castillo is 48, reader is 25. age gap. cheating. f!reader. partners dissing. oral sex (f! and m! receiving). unprotected piv. creampie.

w/c: 9k

Harry Castillo takes many things in life very seriously.

That’s an essential trait when you're sitting in the executive chair of one of the largest construction companies in the United States: being sharp, meticulous, and systematic is as mandatory as a contractual clause imposing penalties for breach.

But there are two things Harry is even more serious and methodical about.

The first: every single one of Harry’s suits is custom-made by the son of the same tailor who once dressed his father and grandfather. Even if a ready-to-wear suit fits him perfectly, it must go to the tailor, even if it’s just to add a single stitch to the inside pocket.

The second: his wife must receive a gift on every single occasion that concerns her or their relationship.

You keep a calendar on your computer solely for this purpose. Her birthday on June 17th, their first kiss anniversary, the day he asked her out, their official anniversary, the day he proposed, their wedding anniversary, Dalilah the Poodle’s birthday.

Yes, there's even an anniversary for the first time they slept together, on September 19th.

And on all these dates, a gift must be sent to her, signed from Harry. If not, she’ll make his life a living hell, and he’ll spiral into one of those gloomy funks for at least three days: always polite, but with short answers and a stone-cold expression. And you hate seeing him like that.

Despite your color-coded calendars and hyper-organized schedule, it did happen once, but only because you didn’t know there was an anniversary for the first time Harry said “I love you,” which didn’t happen until February 15th, 2020, even though he proposed back on October 28th, 2019. Ever since, you make sure that expensive gifts are sent either to their apartment or to her law office.

Today is the anniversary of their first fight, and you're at your desk choosing between a bouquet from The Bouqs Co. and a pair of sapphire Spinelli earrings. Or maybe both?

The elevator doors open and Harry steps out, immaculately dressed in a navy suit you bought last week. He's on the phone and looks stressed. You raise your hand to greet him, and the tension in his face softens into a small smile, which is his version of “good morning.”

He walks past you into his office, leaving the door open, which means he’ll be back in a moment to give you a proper hello.

Harry Castillo’s office is on the top floor of the Castillo Construction & Co. headquarters. Behind your desk, the company’s initials — CCC — are elegantly embossed in gold on the wall. The reception décor is all rich, dark wood — on the wall panels, desks, and on the frames of the chairs in the waiting area. Gold details on the picture frames, doorknobs, and desk edges offer a refined contrast.

It’s beautiful, but a bit dull, so last year, you convinced him to add two dragon trees near the elevator. They gave the space a touch of life, even if he insisted he didn’t like plants in the office.

In the end, he liked it. You know he did.

Being Harry’s executive assistant for the past four years, since you were a twenty-one-year-old fresh out of college, means you sometimes read him better than you read yourself. Your therapist says that’s not healthy, but you like knowing his routine, especially because you’re the one who plans it. You like being his emergency contact, having access to his passwords and bank accounts, being his legal proxy with signing authority.

So, personally, you think your therapist is mistaken.

Ten minutes later, as you confirm your choice of the Spinelli earrings with Harry’s personal shopper, your boss reemerges from his office.

He’s taken off the blazer, and his white shirt sleeves are rolled up, revealing his expensive watch and strong forearms.

“Good morning,” he says with a small smile, leaning casually against your desk. “Did you have a good weekend?”

And here comes the inevitable truth: you are terribly attracted to Harry, which cannot be healthy. Having feelings for your boss, who gives you tasks and commands, kills any remaining instinct for self-preservation.

But God, how could you not? Everything about him pulls you in. The physical traits, the personality, the mind. His strong arms, neatly trimmed beard and mustache, kind brown eyes, tailored clothes, manners, scent, intelligence.

Just the other day, Harry mentally calculated the average profit margin Castillo & Co. made over a five-year period because the financial report hadn’t included it, and then estimated the net return percentage; all in his head. It was the sexiest thing you’d ever seen.

You’ve lost count of how many times you’ve thought of him while with your boyfriend, fully aware of how wrong that is.

“Good morning, Harry.” That’s another privilege: calling him by his first name, while everyone else calls him Mr. Castillo. “I finished watching Russian Doll on Saturday.”

“Yeah? Did you like it?”

You nod, excited.

“Yes, it’s great. You have to finish it.”

Harry gives a quiet grunt.

“I know… But I get home and just crash,” he says, clearly disappointed with himself. You offer an empathetic smile. “I’ll try harder,” he adds, before shifting topics. “I have a meeting at eleven. Can you come with me?”

“Just a moment.”

You open your planner while Harry watches, and you try your best to focus on the color-coded blocks. You have a meeting with the finance team to review some items for Harry, but you can reschedule.

“I can go.”

“Thank God. I’ll need your notes.”

You tap your fingers against your forehead in a playful salute, and Harry smiles before turning to head back to his office. But before he does, he says:

“I like the outfit. Gray is my favorite color.”

He’s referring to your gray pencil skirt and matching halter-style silk blouse.

“Thank you. And I know.”

He smiles, taps his fingers lightly on your desk again, and heads back inside.

And now you can’t focus on anything else on your morning agenda.

The eleven o’clock meeting is at the headquarters of a partner company just a few minutes from Castillo & Co.’s office. Already in the building’s lobby, Harry walks calmly beside you as you head toward the elevator. You’re carrying the leather folder with your iPad and a notepad for Harry, who insists on handwritten notes.

“Did you see how many plants are in the lobby?” you ask as you both stop in front of the elevator, side by side. His security guard stands just behind you, discreet but alert.

“Don’t start,” Harry replies without taking his eyes off the elevator doors. It’s always curious how his expression changes when you’re in public. “You already put two plants on our floor.”

You find it incredibly endearing when he says “our floor.”

“It’s not enough. I’m still planning to sneak one into your office.”

The elevator doors slide open and you both step in. Harry presses the button for the twentieth floor, and you lean against the glass wall at the back of the elevator as he leans in to whisper:

“And then you’ll swing by HR to pick up your termination letter.”

By the time you reach the twentieth floor, where the meeting will take place, there’s still a slight smirk tugging at your lips.

The receptionist at the main desk takes one look at Harry and immediately stands, adopting a posture you’ve come to recognize as reserved only for partners and high-level associates. You yourself soften your voice and demeanor as part of this same executive persona.

You and Harry are led down a long, white hallway with the sterile atmosphere of a hospital (which you hate) until you reach the meeting room. Harry lets you enter first, his hand resting lightly at the small of your back to guide you in.

Inside the glass-walled boardroom, seated at an oval table, are five men and two women. All eyes turn to you, but quickly shift to Harry as he enters the room, already unbuttoning his jacket.

“Please, don’t get up,” Harry says right away, raising his hand palm-out as if to stop them from standing to greet him. Harry hates shaking hands with that many people. “Don’t mind me,” he adds, scanning the room for a free chair. Only one is available. “We’ll need one more chair. I brought my vice president with me.”

Harry is ridiculous. He always introduces you as his “vice president” in meetings like this because, for some reason, if he says “assistant,” the respect people show you is just surface-level, barely polite enough to keep Harry from getting angry. Bunch of assholes.

Someone quickly slips out to fetch an extra chair, but in the meantime, Harry’s hand returns to the small of your back, guiding you to the only available seat at the head of the table, all eyes in the room following the two of you.

Realizing what he’s doing, you whisper:

“Harry, I’m not—”

“Sit,” he cuts you off with just one word, and it leaves no room for argument.

You obey, sitting in the only chair, while Harry stands behind you. With no other option, you slide into your businesswoman persona, straighten your spine, lace your fingers on the table, and meet the stares of the executives around you.

Moments later, someone wheels in another chair for Harry, placing it beside you.

The room falls silent until Harry, now seated and relaxed, says simply:

“So?”

And the show begins.

The goal of the meeting is to convince Harry to invest in the revitalization of a hotel in Madrid, Spain, currently owned by a chain undergoing judicial reorganization. Their last hope is to reopen the hotel, which has been closed for the past ten years, and Harry’s investment would signal a vote of confidence, seen as there’s no guarantee of return for Castillo & Co.

The chain’s administrator, a short man in a tight suit, is in the middle of a PowerPoint presentation showing 3D renderings of the hotel lobby, complete with bronze detailing, when Harry lets out a dramatic sigh and raises his hand.

The man immediately falls silent.

“It’s a good presentation,” Harry says, and you pause your note-taking on the iPad. “But this isn’t what I came to see. Honestly, I’m not the one you should be showing pictures of architecture and interior design to.”

The silence is so tense you could hear a pin drop.

“So far, not a single reason has been presented to me that justifies why CCC should invest in the Madrid hotel,” Harry continues. “Has no one conducted a financial risk analysis? Or at the very least, looked at the average returns of similar hotel chains in the same area?”

“Mr. Castillo…”

“With all due respect, Mr. Edwards,” Harry cuts in again, “my question is simple: was such a study conducted?”

The administrator opens his mouth, likely to offer another flimsy excuse, but this time, one of the women at the table responds:

“Mr. Castillo, we will immediately arrange for a study addressing those questions.”

“You’re asking for more time?” Harry asks, his voice calm, not the slightest hint of aggression, yet somehow that calm makes it even more intimidating.

The woman, to her credit, is brave enough to admit:

“Yes, we are.”

You glance at Harry. He’s tapping his pen against the leather folder he hasn’t even opened. When he stops, it’s to let out a small sigh, as if being in that room is as irritating as a speck of dust in his eye.

“I started construction on a multi-business complex in Madrid last year, and had the bad luck of launching the first month of works right when construction costs in Spain hit a historic record. 117.6 points on the Eurostat index,” he sets the pen down and laces his fingers together, commanding the entire room with nothing but words. “Even with that spike, the real estate market in Madrid is growing,” he glances your way and says, “Miss?”

Of course you remember. You were the one who researched it.

“Seventeen-point-five percent increase last year alone, with a forecast of another four to five percent this year,” you say.

A flicker of pride crosses Harry’s face — but he stays impassive.

“Seventeen-point-five percent,” he repeats, whistling softly in admiration before turning his gaze back to the group. “That’s a lot. Could that offset the budget blowout we’ll likely face by the end of construction in three years? What I do know is that my contract with the buyers of the complex units includes ongoing monitoring of economic indicators and adjustment clauses, because the project team, who are very competent, accounted for all of that. And I only work with competent people.”

More silence.

Harry concludes:

“I expect a study of that level within one month. If you’re not able to deliver that, I kindly ask that you refrain from sending me any more investment proposals.”

Harry stands, and just like that, the meeting is over.

It’s past 7 p.m. when Harry steps out of his office and walks toward your desk.

Under the desk, you’ve already kicked off your heels, and your stocking-covered feet rest softly on the carpet. Your hair is tied up in a bun that probably looks tragic by now, but the kind smile Harry sends your way isn’t one of someone looking at a disaster.

Then again, his hair looks a little tousled too, like he’s run his fingers through it more times than he should’ve.

“What are you still doing here?” he asks, leaning on your desk. He sounds nothing like the man who tore through a room full of clowns earlier in the day.

“I need to go over the spreadsheet the finance team sent me.”

“They sent it late?”

“No. I’m reviewing it late,” you admit, lowering your voice to a whisper and leaning in like you’re telling him a secret. “But don’t tell my boss or he’ll fire me.”

Harry plays along, whispering back:

“A corporate scandal.”

The grin you flash him is ridiculous, and so is the flush that warms your cheeks.

“Still got a lot to do?” Harry asks. You nod regretfully. “Have you eaten?”

You shake your head.

“Alright. I’ll order dinner for both of us. The usual?”

The usual means the Lasagna della Mama Rosa from Piccola that he always gets on late nights like this.

“The usual. Thanks, Harry.”

He ignores your thanks, as always, and heads back to his office. Halfway there, still facing away from you, he asks:

“Want a ribeye? I’m about to beg for one.”

“Rare.”

You can practically hear him rolling his eyes.

“Obviously.”

Thirty minutes later, you go downstairs to pick up the food, paying with Harry’s card. When you return, you head straight into his office.

Harry is at his desk, eyes fixed on the screen. His tablet shows a few graphs, and beside it, his phone is on speaker. He’s talking to his wife, and you pretend not to hear as you walk to the lounge area in the corner of his office, where there’s a leather couch and a coffee table big enough to fit all the food he ordered.

You slip off your shoes before stepping onto the rug and kneel to unpack the takeout bags on the table.

“...because I told her we’d both go with them,” his wife says over the phone, sounding upset. “I can’t back out now.”

“The problem is that you confirmed without even asking me.”

“I thought, as your wife, I could make one tiny decision for the both of us.”

Your brows lift.

“That’s not the point,” Harry says, calm but clearly tired. “The point is you planned a two-week trip out of the country without consulting me. I can’t reschedule twenty meetings or delay fifty different deadlines tied to the 72 active builds I’m overseeing.”

You walk over to the minibar in the corner and grab two sparkling waters and a couple of glasses.

She fires back:

“You could at least try to spend more time with me.”

“You’re being irrational.”

“You drive me crazy!” she yells. “Always with your robotic tone, your charts, your stats. For God’s sake, can’t you be spontaneous for once in your life, Harry?”

You turn to Harry and start to gesture that you’ll leave him alone, but Harry points directly at the lounge area, more specifically, at the table, silently instructing you to go back and stay there.

“You knew who I was when you met me,” he says into the phone, still looking at you. “And I’m not saying that as an excuse for never changing. I’m saying that you need to think about my work before making impulsive decisions.”

She hangs up on him.

You quietly return to the seating area and sit down on the rug, feeling a bit awkward. Seconds later, Harry joins you, settling on the opposite side of the table.

“Smells good,” he says as if he hadn’t just been in a fight.

“Mhm,” you hum, staring at the lasagna in front of you. The smell of melted cheese makes your stomach grumble, but before picking up your fork, you murmur, “I should’ve asked if I could come in. Sorry for overhearing.”

Harry hands you the container with your steak and opens a bottle of water, pouring it into both glasses.

“You know the passwords to my cards and accounts, the backup clouds for the entire Castillo company. My life’s in your hands. It’s not like I have anything to hide from you.”

It’s so satisfying to hear that. Your therapist is going to have a field day.

“You don’t, but maybe your wife wouldn’t love sharing her privacy with your assistant,” you say, mostly because it’s the right thing to say — not because you believe it.

He shuts that down quickly.

“What about your boyfriend?”

“What about him?”

Harry looks up as he takes a bite of lasagna. You pick up your utensils too.

“Is he okay sharing you with me?”

Your hands freeze mid-motion.

“He…” your voice cracks, so you try again. “He knows how much I value my work.”

“Of course.”

The steak is perfectly cooked, tender and rare. To escape the sudden tension, you put on a little show, leaning back dramatically on the plush Nina Magon rug as you chew a piece of meat.

“This is the best steak in the world,” you mumble with your eyes closed. “I’d work overtime every day if this was the reward.”

Harry lets out a low, amused laugh.

“That good, huh? You’d give up sleep for it?”

You hold up a thumbs-up. His laugh grows.

“You should come in later tomorrow,” he says as you sit back up. “That’s me speaking as your boss.”

“I have an eight a.m. meeting.”

“With who?”

“The marketing team.” You already regret it just thinking about it. “Your personal branding, actually. Someone from Forbes wants another interview.”

“Again?”

“Yes, Mr. Castillo. Again. That’s what happens when you’re running one of the world’s top construction firms at forty-eight.”

“Good line. You should pitch that as the interview opener.”

“I will.”

You eat in silence for a while. You take a moment to admire the New York skyline through the huge windows behind Harry’s desk. He likes to keep the lights dim when working late, and the atmosphere feels perfect. The basil lingering in the ragu, the scent of grilled meat, the view of the sprawling city.

Harry sitting across from you. The two of you sharing dinner, like so many times before, and for a moment, it feels like this could be your actual life.

“I can take care of things if you want to go on that trip,” you say, because apparently, your brain-to-mouth filter breaks down when you’re full.

“I know you can.”

“Why not take a vacation?”

“Because I don’t want to,” he says, and you don’t flinch. You’re used to those answers. “I don’t want to travel with the people involved. She knows that. And I have responsibilities.”

“Got it,” you say, leaning back on one hand. Harry watches you. You notice his rolled-up sleeves, the open collar of his shirt, and decide to confess: “I really get it. My boyfriend wants us to go to Bora Bora at the end of the year with two other couples. I can’t stand them.”

“Really? Why?”

“They go to bed at eight. Their idea of being ‘naughty’ is drinking one glass of wine with dinner. Can you imagine that in Bora Bora?”

“Definitely not. Waste of money.”

You snap your fingers and point at him.

“Exactly what I said!”

“You’d like Bora Bora. Rum, sun, and all the shrimp you can eat,” he says, raising his eyebrows. “Might be worth leaving the friends behind and going with your boyfriend.”

“My boyfriend also goes to bed at eight.”

Harry’s face says it all, and so does his smile. He finishes his last bite, scoots back on the rug with his water in hand, and leans against the couch. You do the same, sitting beside him, both of you stretched out in that familiar silence of people who’ve just eaten well.

“Do you two live together?” Harry asks. You shake your head. “How long have you been together?”

You do the math.

“Three years and two months.”

“Has he proposed?”

Straight to the point, as always. Instead of answering, you say:

“Can I grab a ginger ale?”

“You don’t have to ask.”

You walk over to the minibar, grab the can, and come back, fully aware of Harry’s eyes following you the whole time. As you crack open the can, you answer:

“He proposed at the beginning of the year, but I said no. For now.”

“Can I ask why?”

You shrug.

“I’m not really sure. I think a proposal should make you excited about the future, but I didn’t feel that. I felt trapped.”

“I see.” Harry studies your face like he’s searching for something. “I don’t think I felt excited about the future either when I proposed.”

“You love your wife.”

“Do you love your boyfriend?” he returns.

“I do.”

“Okay, but?”

“There’s no but,” you say. “I love him. I love our routine. It’s comfortable.”

Harry is silent, but his expression says he doesn’t buy it.

“Harry.”

“I didn’t say anything.”

“You didn’t have to,” you reply, shifting to face him. ��I love him, but I don’t think I’ve ever been in love with him. No butterflies, no excitement, no stomach-flipping moments.”

“That’s anxiety, not love. Love should be calm.”

“Maybe.”

Silence again. You look out the window. He looks at you.

“I was going to file for divorce last year,” he says suddenly, and it feels like a punch in the stomach. “My therapist told me to wait six months, so I wouldn’t do it in the heat of the moment.”

You’re speechless. He unclasps his watch, slowly continuing.

“I know there’s something wrong with my marriage when I’d rather stay here than go home. I should want to get home to see her. But I don’t. And I know that’s not fair to her either.”

He sets the watch down on the coffee table, next to the empty containers, and rubs his wrist. The hands on the dial show 8:20 p.m.

“I’m sorry,” you whisper.

“Not your fault.”

As he says this, Harry crosses his left arm over his chest to press his right shoulder, wincing slightly.

“Your shoulder okay?”, you ask.

“Pulled something at the gym this morning. Been bothering me all day.”

Before you can even think through the consequences, you offer:

“Want me to press on it a bit? Maybe it’s just tension.”

“Isn’t that a bit outside your job description?”

“I won’t tell anyone.”

Harry smirks and shifts, turning his back to you and giving you space to move closer.

There’s something different about today. You’ve never touched Harry like this before. At most, there were brief handshakes or polite taps on his arm, but now you’re kneeling behind him, pressing your fingers into his shoulder in what feels like the most intimate gesture of your life.

His muscles are rock solid.

“Jesus, Harry. I’m booking you a session with your massage therapist.”

Harry leans forward slightly as you apply more pressure on the tight traps and neck tendon, and for a second, your mind slips to a criminal thought: what he must look like under that shirt.

“Please,” he says, replying to your earlier comment. Then he grabs your hand and places it exactly where it hurts. “Harder, please.”

You press. He lets out a satisfied murmur, and without thinking, your fingers slide under his shirt where it’s already unbuttoned. Warm skin meets your touch, and you feel him stiffen just a little.

“This okay?” you ask.

“Yeah. Keep going.”

You hold one shoulder steady and massage with the other hand under the shirt for a few more minutes.

“If I gave you a raise,” Harry says, “would you become my full-time massage therapist?”

“I don’t even know what I’m doing.”

“And it still feels fucking incredible.”

He never swears around you. Or anyone. Hearing him say that makes the moment feel even more charged. Strangely, it encourages you. You press harder, still behind him, both hands now working the tension from his shoulders.

Then Harry reaches back and takes your left hand. His thumb brushes lightly over your ring finger, and your breath catches.

“There should be an engagement ring here.”

“Maybe.”

“If you get married, would you still work with me?”

“Yeah. I have Stockholm Syndrome,” you say, shifting your position and stretching one leg beside his body. He lets go of your hand, and you go back to massaging, now reaching the base of his neck. Goosebumps rise under your touch. “I could never live without you barking twenty report requests a day.”

“I’m not that bad. I’m nice to you.”

“You are.”

God. His scent is going to kill you.

“You know what the finance team says about us?” Harry starts. You hum, prompting him to go on. “They say you and I are having an affair.”

“Marketing, too. Pretty much the whole company.”

“What? Why?”

Maybe because you turn into a puddle around him.

“Because you pay me more than anyone else,” you say simply. “And I get privileges and people notice. Of course they’re going to think we’re sleeping together.”

“You don’t care?”

“Maybe I’d care if I worked on one of the lower floors. But here? Not a chance. Let them envy me.”

Harry chuckles, shoulders shaking, and rests a hand on your shin, right over the tights. That touch is new too, and, once again, you freeze.

“I know you pay me well because I’m indispensable,” you continue. “Which is very satisfying.”

“So when we stay late working together—”

“Yes,” you answer before he finishes. “They probably think I’m bent over your desk.”

Harry turns to look at his desk. For one second, you both know exactly what the other is imagining.

“Interesting,” he says slowly. “Has anyone ever said anything to you?”

“No. No one’s crazy enough to say anything to the boss’s supposed mistress,” you joke, but the line falls a bit flat, so you quickly add, “According to their little narrative, I mean.”

The awkward moment is cut short by a notification sound from Harry’s computer. You both look toward his desk, and he groans:

“I hope that’s the report from the Chinese investors. They’re three days late.”

He starts to stand, wincing again because of his shoulder, but you place a hand on his arm and get up:

“I’ll check it. Stay put, old man. Even standing up seems like a challenge for you right now.”

“You just got a 10% pay cut.”

You make a “blah blah blah” gesture with your hand and head to his desk, settling into the chair that’s more like a plush couch. On the screen, there’s an open chart, but you quickly move to his inbox.

The latest email is from someone named Yijun, and there’s an attachment.

“You got it,” you say. “Want me to reply?”

“Acknowledge receipt and say I’ll get back once I’ve reviewed the data.”

You begin typing the reply, carefully channeling your best Harry Castillo voice.

Through your peripheral vision, you catch Harry leaving the floor and settling into the leather couch with a satisfied murmur.

“Best regards,” you read aloud, finishing the email. “Harry Castillo, CEO of Castillo & Co Construction. Sent. Done.”

As you minimize the email window, another one pops up. It’s a pre-filled PDF titled “divorce agreement.” You shrink that window as if it had burned your fingers, only to reveal Harry’s personal inbox behind it.

The last message is from his lawyer. You catch a glimpse of the words “as requested,” “speak with her,” “assets,” and “properties” before closing everything immediately.

There’s a knot in your throat as you stand and silently walk back to the lounge area while Harry watches you. He’s left space beside him on the couch, and you settle there, folding your left leg underneath you.

You’re so close that your knee grazes his thigh.

“I sent it,” you say.

“Thanks. You can head home. I’ll stay a little longer.”

“Avoiding your wife?” He doesn’t answer, and honestly, silence is the wiser choice. But you’re not wise. “Can I ask you something?”

“I might not answer.”

“Fair.” You hesitate. “Swear you won’t fire me?” He still says nothing, and you let out a breath, trusting that you won’t be jobless tomorrow. “Is it true you had a thing with the finance manager?”

Harry’s response is a look of disbelief, as if you just told him the strategy department was considering investing in a country undergoing an economic collapse.

“Where’d you hear that?”

“People talk.”

He rolls his eyes.

“Right. And people also say you and I are having an affair, but that’s not true, is it?” If anyone else had used that tone, you’d probably shrink in your seat. But this is Harry. His stress never goes beyond sarcasm—at least with you. “Of course it’s not true. You really think I’m the kind of boss who sleeps with an employee?”

That silences you, and you’re not even sure where this sudden wave of disappointment comes from. It makes you painfully aware of your place in the company. Despite the trust, the passwords, the confidences, in the end, you’re the executive assistant. Nothing more.

“I don’t” you say finally.

He laughs, incredulous.

“Why do you sound disappointed?” he asks. And at this point, you don’t even know what to say, so you start putting on your heels instead, but Harry is faster. “No, no… Hold on.”

“Do you need anything else?” you ask politely, your left foot already in the shoe.

Harry freezes, eyes locked on you, and you freeze too.

“I have my morals,” he says.

“I know that,” you shake your head slightly, as if trying to hear him better. “Sorry, what do you mean by that?”

“I mean I have my morals, and that’s why I’ve never tried anything in here with the one person who makes me want to, especially because she’s my fucking assistant.”

God. You freeze, heart racing. Your mind latches onto the tense of the verb.

“Makes? Present tense?”

His quiet laugh is almost bitter.

“Unfortunately,” he says, settling back into the couch. “My father raised me right. I have morals, I respect my wife, and I care about my reputation.”

You drop the shoe again and turn to him. Your question is clear, firm:

“Even on nights like this one?”

He says your name like a prayer, rubbing his face with one hand.

“Don’t do this.”

That quiet, simple plea brings you crashing back to reality for the thousandth time. You whisper an apology just as softly, pick up your heels again, and before you can put them on, the leather cushions shift beneath you.

That’s the only warning you get before Harry is close behind you, his hand gently gathering your hair and moving it over your right shoulder to expose your neck.

“I have my morals,” he repeats, coming closer. “Don’t you?”

You think of your boyfriend, and how sweet he is to you. Your mind conjures up images of happy moments, trips, dinners, gifts, and you know you can’t just shove those into a box and lock it away for a few hours. That’s not how it works.

But the way your stomach knots with Harry’s closeness shrinks all those memories down like a sheet of paper folded over and over. They’re still there, but small. Insignificant.

“I do,” you say, because it’s true. “But I can live with that.”

“I don’t know if I can,” Harry murmurs the way he always does when something matters, as if tasting the words.

“If you’re just going to feel guilty—”

“I’m not talking about guilt,” Harry interrupts. And then his hand is on your stomach, pulling you back toward him with one decisive motion that makes you gasp. “I’m saying having you just once wouldn’t be enough.”

“Well, it’s going to have to be.”

At the very first touch of Harry’s lips on your neck, your entire body feels like it’s catching fire, every nerve alive with want, your hands clenched tightly on your thighs. It’s as if every hair on your body is standing on end.

“Did you forget I’m the one giving orders here?” he says. “Once isn’t enough.”

“Is that a command?” you challenge.

Harry’s mouth trails down to your throat, leaving open, wet kisses on your sensitive skin.

His fingers glide lightly to your breasts, the tips barely grazing your nipple through the silk of your blouse. The friction of the fabric makes you arch into his touch so slow and torturous it nearly drives you mad.

“If only you actually followed my orders,” Harry murmurs.

“Of course I do.”

“Yeah?” He kisses the corner of your mouth, pausing just to say, “Then get on your knees for me.”

You shift on the couch to face him, and suddenly, it all feels terrifyingly real. The weight of what you’re doing crashes into you like a slap across the face, because he’s right there, wedding ring on his finger and lips still flushed red.

But unfortunately, it’s not enough to make you stop.

“I want a kiss first.”

Harry parts his legs, giving you space, and you rest one knee between them on the couch, moving in closer to sit on his thigh. You run your fingers along his cheeks, his beard, the collar of his perfectly white shirt. It’s the first time you’ve touched him like this, and you’re certain your gaze gives away more than you want, because there’s a softness in the way Harry pulls you closer.

You’ve caught yourself wondering what kissing him would be like, even during office hours. You’ve seen him kiss his wife before, but it was always just polite pecks, the kind of affection acceptable under New York’s high-society scrutiny.

But nothing could have prepared you for how naturally your lips fit together, or how good it feels. It’s even better than you imagined, just like the rush of doing something so wrong, yet so irresistible, precisely because it’s forbidden, and everything you’ve secretly wanted.

Harry’s hands slide to your waist, deepening the kiss, and yours go straight to his hair, already messier now. The moment his tongue touches yours is the same moment his hands slip beneath your skirt, lifting the fabric as they go.

He finds the lace tops of your stockings, held in place by a garter belt. His hands go straight to your ass, gripping tightly as if it’s instinct.

The curse he whispers makes you smile.

“Take off the skirt and blouse. Get on your knees,” he says, cupping your face and pressing one more kiss to your lips. Then, with a whisper: “Please.”

Hearing this man plead is a dream come true, which is exactly why you nod right away and walk toward his office door.

You close it. Lock it. And as you return to him, you unzip the skirt and slip off your blouse, leaving it behind in your path. The air conditioning makes your nipples hard and sends chills across your skin, but Harry’s gaze, now seated deep into the couch with legs parted, more than makes up for the cold.

Next goes the skirt, and now you’re standing before him in just your stockings, panties, and garter belt.

His lips part as he draws in a deep, appreciative breath, eyes trailing slowly up your body. It’s almost as if he’s touching you with his stare. His hand goes to his tie, loosening it as you sink to your knees.

With your hands resting on your thighs, you watch as he pulls the tie off (the one you bought last month) and undoes the top buttons of his shirt. Next comes the belt and then the button on his pants. Harry leans forward slightly, legs still open, and pulls himself free from his boxers.

Despite the curiosity and heat flooding through you, you keep your eyes locked on his until your tongue brushes the tip of his hard cock. Harry exhales sharply, eyes fluttering shut, and there’s a quiet power in watching a man like him unravel — even just a little.

That alone is enough to make you take him fully into your mouth, lips closing around his thick shaft, sinking him deep.

It earns you a low, guttural curse.

Harry gathers your hair in one hand, holding it tight at the base of your neck. You have one hand on his thigh, the other stroking what your mouth can’t reach, and for a few minute, you lose yourself in the weight of him on your tongue, in his taste, his scent, the sounds he makes just for you.

And then just one question slices through the haze:

“What would your boyfriend think, seeing you like this?” Harry asks, his voice so polite it almost clashes with what you’re doing. He pulls your head back, letting his cock slip from your mouth, dragging the tip across your lips like he’s marking you. “On your knees for your boss. Do you suck his cock this well too?”

You narrow your eyes.

There’s probably an unspoken rule about not mentioning spouses or partners during moments like this. The act is already betrayal enough.

But if Harry wants to play that game, you won’t back down.

You rise slightly on your knees, aligning yourself so he can press his cock between your breasts, and you reach for his mouth to whisper:

“And do you get this hard when it’s your wife sucking your cock? Because if you did, you’d probably want to be home right now.”

Harry smiles against your lips and kisses you again as you climb onto his lap, and he remains silent.

“Let’s go all the way,” you say, because you’re far too wet to let this go to waste. “Right?”

“Right,” Harry answers without hesitation. “No turning back.”

“Do you want to?”

He slips his hand into your panties and finds so much wetness that his fingers glide immediately. His answer comes when he lifts the same fingers to his mouth, eyes locked on yours.

That makes you rush to unclip the garter belt and slide off your panties, tossing them aside. Harry gets the message and starts striping off his pants and shirt. And suddenly you’re on your back with Harry’s heavy and sturdy body on yours, skin on skin.

Harry rolls down your stockings in one smooth, hurried motion. You wrap your thighs around his hips.

“I don’t have a condom,” he says, and God, if eyes could beg, his would be on their knees. “It’s not like a married man needs to carry one around.”

“I printed your test results last week. And I don’t have sex without a condom…” you begin—and then add, “…with my boyfriend.”

He gets it.

“Can I?”

“You can.”

Harry doesn’t even glance down as he guides himself inside you, keeping his eyes on your face, your mouth, his own opening bit by bit while sinking into the wetness. When he’s fully buried, you have to shift your hips to adjust to his thick length.

“Just a second,” you whisper, wrapping your arms around his shoulders. He nods, and you take the moment to ask, “Had you imagined this before?”

“I don’t know how to answer that without sounding like a pervert.”

You run your thumb across his eyebrow, studying his features in the dim light of the office.

“Would it make you feel better if I told you I’ve imagined you while fucking my boyfriend?”

Harry raises an eyebrow.

“I want details.”

“Earlier that day you and I were at a meeting. You did some absurd calculation in your head, and it made me wet. So I went home and…”

“Fucked him while thinking about me,” he finishes, smiling. “Filthy mouth.”

When you keep staring at him, silently asking for his turn, Harry sighs.

“Of course I’ve imagined it. Every time we stay late together, or when you wear that damn red dress and walk into my office, and especially when you put arrogant assholes in their place. You drive me insane.”

You reach between your bodies, your fingers trailing along where you’re joined, circling the base of Harry’s cock. He jerks his hips reflexively, breathing out a soft moan.

“And…” you press.

“And sometimes I dream about you and wake up so fucking hard that…” Harry begins to move his hips slowly when you give him a nod. The thrust is deep, slow, excruciating, and he fills you entirely. You almost miss his next words:

“…I wake my wife up and fuck her.”

“While thinking of me.”

Harry grips your hips and covers your mouth with his:

“While thinking of you.”

Your mouths open into a kiss that matches the way he fucks you: raw, urgent, drenched in tension. Every thrust hits something deep inside you, something you’re not sure anyone else ever will again. You cling to his shoulders, resisting the urge to claw at him, lifting your hips to match his rhythm.

You’re soaked, so much it’s nearly embarrassing, and you’re certain Harry’s lap is drenched with it too. As his movements grow more erratic, you slide a hand between your legs.

Harry catches your wrist, guiding it back to his shoulder.

“No, no… You’re gonna come on my mouth later.”

Well. Okay.

Harry shifts to sit back on the couch, one foot planted on the floor, the other tucked under his leg. He pulls you into his lap again, and this new angle makes him reach deeper, every little shift filling you completely. When he's about to come, he grips your waist tightly to keep you still and thrusts harder, driven by your moans, his mouth open against the space between your breasts."

“Can I come inside?” Harry asks, holding you firmly.

“Please.”

He groans, wrapping his arms around you, and just a few more thrusts later he’s pulsing inside you, breathing heavily against your skin. The warmth floods you in a way that makes you throb for your own release.

“Harry, I need to—”

“I know.”

You’re not sure how it happens so quickly, but in the next second he’s back on the couch, and you’re straddling his face. Then it’s his mouth, his lips on your aching clit.

You grip his hair and glance down, meeting his gaze. Your whimper turns into a moan as he drags his tongue along your folds, tasting both of you, and returns to sucking that overstimulated spot.

“Stick your tongue out,” you beg. “Please—”

He does, and you immediately grind against it, whispering Harry’s name over and over like a prayer.

It hits you like an earthquake. So sudden, so intense that your whole body trembles on top of him, and for a split second, it feels like you forget how to breathe. When you come back to yourself, you’re sitting on his chest, and Harry’s wiping his beard with the palm of his hand, a crooked little smirk on his red lips.

You look down at him and say:

“We’re going to hell.”

He wraps his arms around you and sits up, keeping you in his lap.

“I’m an atheist,” he says, kissing your shoulder. “So… okay.”

“Okay.”

“And now?”

“Now,” you say slowly, cupping his face and making him look at you again. “This never happened. We go back to our lives like nothing ever did.”

Harry sighs your name.

“You say a lot of smart things. That’s not one of them.”

You pinch his cheek, offering no reply, and slip off his lap to gather your clothes from the floor. Your stockings, panties, skirt, and blouse. When you return to the couch, Harry’s already pulled on his boxers and pants, so you sit next to him to do the same.

The entire process of getting dressed again is done in silence, and you’re not sure what you feel: shame, guilt, some strange sense of calm… The only thing that doesn’t hit you is regret — and that makes you feel guilty too.

As you’re slipping on your heels, Harry says:

“It’s only nine-forty.”

“Hm?”

“We still have two hours and twenty minutes before the night’s over. And I’ve got an empty apartment about twenty minutes from here.”

You look up at him, and he adds:

“If tomorrow we’re going to pretend this never happened, we might as well make the most of it tonight.”

You know it’s a terrible excuse. You know that tomorrow neither of you will be able to pretend this didn’t happen. You don’t know what comes next, and the ring on Harry’s finger sits like a weight in your gut, but you’re not a good person.

You lied to Harry. Your morals are bent, and even though you’re fully aware of the circumstances, they don’t stop you.

Nothing could stop you from getting what you want. And right now? You know exactly what you want.

“I’ll wait for you in the garage,” you tell him.

#harry castillo x reader #harry castillo #harry castillo imagine #harry castillo fanfiction #pedro pascal x reader #pedro pascal #pedro pascal imagine #pedro pascal fanfiction #mine

4K notes · View notes