#how to remove duplicate rows

[ss-shitstorm]

Sorry the next chapter of Bread is taking so long here's an excerpt

Your almost friend pulls you up the final step, pausing at the railing to allow you to get your bearings. How thoughtful. It's every bit as cheerfully chaotic up here, seats, barstools and stage all occupied by scaled-down bots in varying stages and sorts of intoxication.

Actually, the crowd in front of the stage seems more densely populated then it’s downstairs duplicate, tossing their version of currency at a pink and more pink femme scattering violet sparks as she spins suspended by only her hooked wrist and ankle.

Same shit, different stage. You swallow thickly, following Starscream into then around the crowd, wincing as a rust colored bot spins around on his barstool to loudly wolf-whistle at you two, while his buddy sloughs off onto the floor like wet play dough during his own attempt. Holograms or not, they seem to walk, talk and fuck like their living counterparts, and having a hyper-realistic crowd to practice in front of for the first time ever is giving you the heebie jeebies. Especially since they seem cognizant of your exotic-by-proxy status, prompting hushed whispers and elbow-jabs as you walk by.

“Yoooo is that an organic? Primus it’s an organic.”

“SHOW US YOUR PLUMBUS-!”

“Shut the FRAG up SmackJaw, they don’t all have those!”

“God, did you have to put so many people in here?!” you hiss, watching the minicons in the back rows leaving their seats to scoot closer up front as you join your companion on the stage.

“How else do you expect to get used to it? At least this audience won’t cause a problem.” He illustrates his point by kneeling down on the ledge, which “Smackjaw” is attempting to drunkenly climb, and proceeds to smack him directly in his jaw hard enough he falls backwards onto the floor.

Your own jaw drops in horror. Then disbelief as he stumbles back to his peds, blinks a few times, then goes back to cheering in a repetitive NPC fashion.

“Alright everyone-!“ Shouts Airplane man as he rights himself. “This is a LESSON, not a show. You can stay if you want, our little rookie here would benefit immensely from the pressure if you do-“ he gestures toward your shaking self as one would a frightened rabbit, hopefully not one held over an overpass. “-but they’re NOT exposing their plumbus.”

That’s enough of a deterrent for some, but not all. Smackjaw and a few others stay rooted in place while their peers shuffle to the bar or the back, where someone had unleashed a multicolored glowing beach ball to toss around.

“I can spawn a few more helium lob-balls for them, if you’d like.” Offers your teacher, who’s now leaning against the frontmost pole with his arms crossed.

“I-“ Deep breathes. Deep, deep breathes, until you hyperventilate and pass out. You exhale shakily, biting your trembling lip. “-no that’s….that’s okay.”

“You do realize how low the stakes are, don’t you?” he raises an optical ridge. “You concoct more deadly things in your lab on a daily basis and make a hobby of trying to die. Where exactly do you get off being petrified by a bunch of programs?”

He's right and you know it. But tell that to the part of your brain responsible for social anxiety, public speaking and removing clothing in public anywhere other than in front of Garbage man’s garbage gaze. “I don’t. But it’s…it’s different, okay?!”

“I know it is. Appealing to logic works for some bot’s jitters, but not others. I suppose you fall into the “others” category.” He steps off the pole, over to you and kneels down, much to your confusion. “Sit down for a moment, would you?”

You do as told, sliding into a shaky mess on the floor. “Why?”

“Because I’m giving you a medicinal solution to your jitters.” He says, opening his servo to reveal half of a Valium tablet.

Oh boy. Dr. Feelgood at it again. “That’s gonna take too long to start working.”

“If your INTAKE in the orifice you cram it into, then yes. But I’ve done my research of this substance and its bio-availability to your species. There’s other ways that, while reducing the efficacy a bit, will send it speedrunning into your system.”

You choke on nothing. “I’m…I’m not putting it in my ass.”

“Vector-sigma no! No. Why is it always feces and fecal accessories with you?!” he retches.” That’s not what I’m suggesting.”

“Then what are you suggesting?”

He answers by abruptly closing his servo around the tablet, opening it once more to reveal he’d crushed it to powder. He then procures a thin metal tube like the one you’d seen Knockout use, and offers it to you.

“Insufflate it into your olfactory organ. It should hit in five kliks tops, peak in a quarter of a groon. If you come down while we’re still working, I can give you the other half.”

Understandably, you’ve got reservations. Reservations that are reviewed and disregarded in a manner of moments, because you can’t be wasting anymore time on this. You’re learning to pole dance in a cat costume from a sentient Airplane to save a rabbit from a perverted meth kingpin mayor you now sell alien chocolate narcotics to, and none of those things should have ever come together to make a sentence. If snorting sedatives out of Airplane man’s cupped servos is going to get you done with this thing and back to your other, equally stupid jobs faster, then you’d be even stupider to not do it.

“Okay-“ you say, tube already in your hand as you push the tip into your nostril, close the other one, and proceed to clean the powder out of his hands.

You’d expected it to burn, probably due to the stabilizers to keep it in pill form. You hadn’t expected it to punch you in the back of your mouth through your nose, making your eyes water as it congeals, oozes, then drips down the back of your throat, where it also burns. You take the tube out, groaning, sniveling and clutching your head as you try to get to your feet, only to be firmly held in place.

“Not yet.” He takes the tube, roping his massive arm around both your shoulders like a lead blanket. “Stay put till it kicks in, then several moments after. Once you’re certain the room isn’t going to start spinning, or once it’s stopped, then I’ll help you up.”

You don’t try to argue, waiting impatiently for the familiar, dreamy, I-never-had-any-fucks-to-begin-with- feeling to come creeping up. Or flying-jump-kick you in the dick.

It seems to be a combination of the two; a lucid apathy setting in the precise moment you open your mouth to ask “how long-?” Only to have a “Wow…okay, yeah…wow.” flop out instead as the sensation surges, nearly knocking your seated ass backwards. Your limbs aren’t limp marionette strings this time, but the muscles in your back relax enough your torso struggles to keep you upright.

Fortunately, your lead blanket has equally few qualms about becoming a backrest. He shifts his weight, bracing the arm not slung around you to hold himself upright so you can lean into his chassis.

You wonder how long he’ll bother to stay like this until he gets bored, impatient, or decides you’re gross again and shoves you out of his lap. You wonder what exactly it’ll mean if he doesn’t do any of those things and stays put. You also wonder when exactly the last time you’d felt this at ease with someone, drugs and death machine nonwithstanding. Because despite everything, you’re experiencing a brief, Bodhisattva level of peace.

“I…uh….yeah..s'good.” you begin so very sagely. “I think I’m…ready.” you flit your (only slightly) blurred vision to your backrest’s face. “Thanks for waiting.”

“Don’t thank me yet.” He doesn’t move you, but retracts one of his arms, rolling his neck with a wince. “Not that your minuscule frame could cause any damage, but I’ll need you to return the favor. Sitting here has given me a bit of a crick.”

Blinking not entirely in sync, you crane your head back a bit further than it should go to see him reaching his free arm and servo into his subspace, emerging with a container of dusky blue powder.

“That’s…” you pause, tongue unpleasantly thick and dry against the roof of your mouth. “…that’s not Valium, is it?”

“I’ll consider that a rhetorical query.” He says, sparing you the associated look he’d give if he didn’t. “It’s nucleon nail in freebase form. A bit of a pain to evaporate and salt out of the injector, but far easier to dose out in this manner. Especially if you’re not planning on being unconscious.”

Like your long-suffering, still-recovering B1ll. The same thread of concern unraveled for your assistant tangles for your current companion, though knit with strands of incredulousness. “You’re sedating yourself?” you ask, lolling into the crevice of his side and elbow as his massive-by-comparison form shifts around you to bring the container in front of both your faces. “You’re the teacher and you’re sedating yourself?”

“Firstly, I’m relaxing myself.” He gives the container several firm shakes before popping the lid open. “I’m taking half of a recreational dose, and less than 1/4th of a therapeutic one. Secondly, it’s not just for relaxing. It’s for pain management. One doesn’t live through a war that spans planetary life cycles without incurring multiple injuries, not all of which heal properly or stay healed. Grind-dancing is likely to aggravate at least some of the scars I’ve brought back from the battlefield. He pauses, loosing a bitter growl under his breath. “Or those acquired closer to home.”

He's referring to the maulings your Mastiff dolls out. Both ones you’d failed to prevent, and ones that occurred before your planet hosted sentient life. Your heart tries to plummet, the diazepam slowing it’s fall to a gradual tumble. “I…okay yeah. Sorry.” You blurt out sheepishly. “I’ll raise my hand before I ask another stupid question.”

“Yes, well I’m not sending you to detention quite yet.” He plucks the metal tube still held loosely in your hand, before turning it palm-up towards the ceiling, cupping it in his servo. “Ready to reciprocate?”

You’ve less than zero issues doing that, but the sheer insanity of the situation still gives you pause. Snorting sedatives and alien pain relievers with an alien in a holographic representation of an alien strip club may well be the most ludicrous thing you’ve had happen to you to date, and considering the batshit ordeals you’ve been through and continue to go through in order to protect, serve, and serve your captors fecal-based-hydrocarbons, that’s fucking saying something.

This doesn’t feel like an ordeal anymore, though. In fact, it feels like the exact opposite. It feels special, intimate. The way two beings that genuinely find relief in each other’s presence feel on an excursion planned for exclusively the two of them.

It feels fun.

“Sure.” You hold both hands beneath the container in wait. “I’m guessing the uh…mass displacement doesn’t affect the dosage?”

“Not if I don’t revert to my full height till after it’s been metabolized.” He uses the tube to scrape a dime-sized amount of out the capsule and into your palms. ” Before then, it’ll be reduced to 1/10th of it’s efficacy and I’d get more pain relief from being bludgeoned in the back of the helm.”

There’s probably some fascinating physics behind that. Physics you’re not going to dissect because it falls squarely outside of your jurisdiction of mad chemist and alien cocaine mirror. Instead you stare transfixed, watching the twinkling powder, cool and oddly ticklish to the touch collect in your palms till he closes the lid.

“You really don’t have any reservations about touching organics, do you?” you ask while he cranes his head and neck forward over your shoulders, bringing the tube to his face with one servo, and raising your cradled hands with another.

He grants you a sidelong glance over your own shoulder, lambent Japanese carmine optics narrowing in amusement.

“You’ve already been in my cockpit, haven’t you?” he asks with a grin that makes your lungs stop working. “Were you acutely toxic, I would’ve been poisoned well before now. But honestly-“

He pauses, lowering his helm, shuttering his optics, and vacuuming the powder into his nostril with a soft grunt that sounds the way satin feels. “-you’ve proven to be more of an antidote, haven’t you?”

He lowers the tube and your hands, sniffling incessantly and turning wide, owlishly blinking optics toward the ceiling. At a loss for words, you don’t comment further. Somewhat because that last line was capable of scooping up someone 3 tiers out of your league at any club, alien or no. But mostly because the expression he makes, clutching the side of his face, optics half-shuttered and biting softly into the plush of his metal lips, grants the realization that out-of-your-league someone owns the lap you’re currently sitting in.

Starscream is attractive. You’ve witnessed literally everyone on the ship looking for too long when he walks away, bends over, or puts the "Airplane” in Airplane man and takes off into the stratosphere. And like many ‘isms blessed with their race’s beauty standards by default, he’s also prideful. You doubt he wants anyone beyond the CMO to know he has injuries or pain he’s forced to medicate for. That you do know paints the picture you’ve just witnessed something fairly vulnerable. A vulnerability he’d not only allowed you to see, but trusted you to participate in. Since he trusts everyone in his faction about as well as you do(which, beyond Soundwave and Lazerbeak, is no one), your mutual lack thereof functions a bit like an olive branch.

This whole setup is an olive branch, actually. He’d not only not asked why, but nearly jumped at the opportunity to give you lessons, then dosed out anxiety medication he keeps on hand for you specifically, and was comfortable enough to eat nose candy out of your hands without a second thought. Comfortable enough to leave you lounging against his chassis with his arm slung over your shoulder. To absentmindedly thread his talons through the strands of hair that falls at the nape of your neck. To guide you to the epiphany that, while your attempts to expose the fleshie-fragger your guardians had spoken of hadn’t yielded fruit, they had unintentionally narrowed your search down to a razor-thin line.

A line so thin, perhaps, it could only be traversed by stilettos. Like the ones attached to the disgustingly handsome SIC languidly rolling himself out from under you, getting to his feet, and offering his servo to help you do the same.

"Oh god, it might be Starscream." You think, dawning horror and trepidation freezing in your veins like ice as you take his offered servo and allow yourself to be pulled upright.

“Oh god-” you think again, horror and trepidation thawing to exhilaration as he leads you to the pole, servo squeezing your hand not enough to cause discomfort, but too tightly to ignore. ”-it might be Starscream.”

[jon-snows-man-bun]

By Turns

Chapter Eight

The closer Eris gets to his goals the harder he has to work to keep all plates spinning. Tensions simmer underneath his new alliances, pulling him into the Hewn City where the impact of Rhysand’s rule shapes the future.

Masterlist

Find this fic on AO3

A/N: Chapter contains minor domestic violence, an explicit smut scene, and dubious consent. I tried to make the consent jump out, but the context of the encounter is very grey.

Happy Eris Week to all who celebrate! This is an ongoing work not written for Eris Week, but fits today's theme of Bonds / Bargains.

Thanatos’ request for a meeting hadn’t arrived by the traditional means, but rather had been slipped to him by one of his spies. Eris normally appreciated the theatrical flair of duplicity, but he found this overly tedious.

He found most of the Night Court dull work, actually, excepting his brother and the little raven-haired female he wanted. Rhysand’s aggressive, arrogant attitude permeated his whole Court, and if the division between himself and the Hewn City wasn’t ripe for exploitation and profit, he would have preferred to deal only with Lucien.

What he had assumed was paranoia from Thanatos by insisting on a secret meeting turned out to reveal another division, which he hadn’t anticipated. He knew the leaders of the Hewn City wanted nothing more than to be free to come and go as they pleased and perennially rowed with Rhysand about it, but he hadn’t expected the steward and the general to be rowing between themselves about the best method to achieve their ends. Keir – already nervous of Rhysand – had wanted to wait for another opportunity to exploit, to leverage Rhysand’s eventual need for the Darkbringers in exchange for freedom. Thanatos, however, had apparently noted his relationship with the Inner Circle and decided to try to win Eris to a different cause.

“Keir would rather await Rhysand’s favour, which we both know is never forthcoming. How long has he maligned you? How long as he maligned us? He is unforgiving once his judgements are set. He does not reverse course. I do not have Keir’s patience, but Keir doesn’t have the courage to act decisively,” Thanatos complained, falling far short of the role of charming emissary. Even Cassian the oaf had done better.

“And the Darkbringers are enough to defeat Rhysand and the Illyrians?” Eris said, gauging the request for Autumn soldiers forthcoming and ready to refuse.

Rhysand was his ally, no matter how uneasy; he needed Night strong enough to counterbalance Beron’s ambitions of expanding into Spring. Eventually, he would need Spring strong enough to deter any advances from the Continent – and scupper any ambitions from Rhysand. He had seemed entirely too comfortable stealing from Tarquin and sending Feyre to topple Tamlin; Eris was deeply sceptical of his ability to remain within the bounds of his own Court and leaving too many Courts weak left an easy opportunity for Rhysand. He had already been too eager to try to dispatch Illyrians into Spring under the guise of stabilising the shaky Court. Prythian worked best when the High Lords tended to their own gardens.

“I don’t need an army and he need not be defeated. Merely convinced. Why content yourself to wait for what you want, when you can reach out and take it?” Thanatos said, and his teeth gleamed white in the gloom. “I imagine you ask yourself the same thing.”

Eris kept his face blank, but his magic simmered under his skin. He was adept at this game of concealing his motivations, but still Thanatos smiled wider, as vicious as a shark.

“Rhysand will use the Darkbringers against you, when it suits him. I promise he will. If you grow too strong, he will invent a cause to prune you down. He grips the City with his left hand, Illyria with his right, and uses both as his blades,” Thanatos said.

“And you wish to remove his hand?” Eris drawled, eyes narrowing. His thoughts a moment ago had run along the same lines.

“Merely the hilt. I would never seek to harm our beloved High Lord,” Thanatos said, utterly deadpan. “You are allies with Rhysand. When the time comes, encourage him to loosen his grip. Perhaps it will become loose enough that the Darkbringers may lend aid to your own soldiers. But if he still keeps his boot on our neck and the wards of the City active, then we cannot leave no matter how much we might wish to lend assistance.”

Eris kept his face arranged in a disdainful frown, but he was turning it over in his mind. He was General of Autumn’s armies, but not all were loyal to him. When he eventually did overthrow Beron, it would likely involve a clash between Beron’s loyalists and his own. It would be the most fraught period of the transition, and a legion of Darkbringers to tip the scales in his favour was no small consideration.

“And are you so sure the time will come?” Eris asked, prodding for details. “He’s ruled for centuries and has done nothing to change.”

“A sword without a hilt is hard to grasp,” Thanatos said, and smiled, full of dark promise. “No power is eternal.”

He watched Eris for a long moment, then sought to strengthen his suit.

“Have the female you like as well, if you wish,” Thanatos offered, almost too casually.

“What use do I have for a female?” Eris sneered, immediately defensive. This fucking place – everything went noticed, every minute interaction. Aisling had already suffered for his attentions. When he sensed her in that corridor, he had decided impulsively to see what he could get away with in the clandestine corner. He was so preoccupied with thinking of touching her perpetually bared waist that the smell of blood hadn’t registered. His stomach had dropped when he thought it may have been her blood that was spilled, but her whispered confession that she had been chastised because of him set his resolve.

Eris suspected someone may have been watching them, or at least would happen upon them, and counted on the information being passed along. They were all afraid of him here. If the Hewn City assumed he was fucking her, he thought it would at least grant her some protection by extension. It was good for them to fear drawing his ire and would let Aisling escape further torment. Her magic was strong and useful, and he had been idly considering ways to use it in exchange for his protection.

He hadn’t anticipated being just handed her, though.

“The usual,” Thanatos smirked. “Aisling. A beauty. I’ll have her sent to you, and you can invent some use for her.”

This place disgusted him. Something dark and angry moved in his blood, even as the thought of her waiting for him in his bed made him simmer. He was a wicked creature at heart, he knew. But coerced… He would offer her the choice, he decided. She could hide behind him if she wished, and he would keep her secret. What she did next under the cloak of his protection would be her business, and none of his concern.

“You honour me,” he drawled.

“I honour your word,” Thanatos said. “We have a glut of females, after the cunt queen and Hybern. She goes surplus. Keep her as a consort here, if you’d like. Or for a night as you please.”

And that was that.

———————

Aisling had broken into a cold sweat when Maeve informed her that a summons demanding she attend Lord Thanatos had arrived. After that miserable afternoon in the throne room she had been all but hiding, focusing on activity in the mine, neglecting all social responsibilities. She had convinced herself everything would be fine, if only she stayed a recluse for long enough; females often disappeared from the public eye for weeks at a time, it was nothing new. People would forget, and move on to the next bit of gossip.

She had been leery of further chastisement from Lord Keir, who was never one to let a female act with any independence and always enjoyed punctuating his point. She hadn’t expected to be called by Lord Thanatos, whom she had never spoken directly to.

The General was sat in a carved wooden chair, studying her with eyes the absolute black of the Darkbringers. Aisling knew enough to wait silently.

“I should have taken you as replacement for the son you killed when you were a child,” he said, finally. “It would have been done that night but I insisted a daughter was not equal to a son, no matter how much gold she would one day possess. I was already troubled with a daughter and did not want another. I refused Keir when he suggested it. More fool me.”

Aisling’s stomach flipped, and she looked up sharply, against her better judgement. She didn’t know that, had never realised that had been an option. The thought alone of living under Thanatos’ heavy hand rather than her own absent fathers made her ill with dread.

Her power had manifested in a sickening rush, all at once like a dam bursting. She had been ten years old and gripped by the magic; it ripped through her in a violent torrent that left her shaking and gasping.

It had found its mark in Thanatos’ younger son, who had the misfortune to upset her. It was a childish disagreement, but he had replicated the example of his father and hit her with a closed fist. She had screamed, and her breath had carried the seeds of a nightmare – he had walked in his sleep, or run in fright, or been trapped in a dream, she wasn’t sure what she had done – but he had leapt from an upper balcony to his death in the stone courtyard below.

The memory of it still left her sick. Aisling had carefully packaged the memory up, sealed inside a strongbox in her mind, and never disturbed it. She wrapped it in chains and sunk it into the black depths of her heart.

“As your own father is dead, it is for Keir and me to arrange a match for you. I am giving you to Eris Vanserra,” Lord Thanatos said, finally. “You’ve already acted poorly around him and in a way that would shame your father. This seems a fitting arrangement, given your behaviour.”

The floor dropped out from under her, and she knew she was gaping. This was a bid to control her – she had slipped their net for too long, had been too brazen with her independence. She couldn’t mask her surprise, brows furrowing.

“To wed?” She asked tentatively, trying to keep pace. She had the sense she did sometimes while dancing, if she was drunk enough and the music fast enough. Certain dances would speed up and up, building to a frenetic tempo, faster than she could keep pace with. If she did not keep her feet moving here, she would soon start getting dragged.

“To do with as he pleases,” Lord Thanatos said, with no small amount of relish. He was basking in this, in bringing her to heel. “You seem surprised. Do you have any objections?”

“I don’t understand -”

Her father was long dead and her mother was not interested in disciplining an adult, so it had been many years since Aisling had been struck. She no longer anticipated it, and so the blow from the back of Lord Thanatos’ hand blindsided her. His hand cracked across her face hard enough to stun her and sent her staggering sideways.

Her face felt hot and tight, and her ears ringing. Her mouth hurt so badly she was certain he had knocked teeth loose, but when she touched her lip, there was only a smattering of blood on her fingers.

“You don’t need to understand, you need to do as you’re told,” Thanatos snapped as she gathered her wits, reining in the temper that flared hot and bright. Had he been waiting for this moment since she was a child?

“A household shouldn’t be without male leadership. You needed a male to guide you and shape you,” the lord tsked, watching as she drew herself back. “You’ve grown too brazen. Vanserra will break you in.”

Aisling tipped her head, playing cowed as she fumed inside. Thanatos studied her for a moment longer, searching for another reason to correct her. Or perhaps he was merely enjoying the moment of justice he had wrought for the life of his son.

“Go,” he finally dismissed her. “I’ll summon you when he is next here. And see a healer so you don’t bruise. He won’t enjoy you smudged.”

Aisling fled before he changed his mind.

Anxiety and anger and desire competed in her the next few days, warring for attention like rowdy children. Eris always made her feel so unbalanced, and ever since he had kissed her, she had turned the moment over and over in her mind and still could draw no conclusions. He was charming then cruel then kind, and she had the sense she was being played somehow, but couldn’t ferret out how.

Part of her hoped Eris had not truly made some agreement with Lord Thanatos, hoping against all rationality that he would not find joy in a coerced consort. He had encouraged her independence in some small way, even if he derided her for it. She wanted him to refuse, on some honour or chivalry, and be….

Someone he was likely not, Aisling admitted, chagrined.

The other part of her – the one that snapped open like a jaw trap when her magic woke in her – was relieved she wasn’t being sent to the temple altar. This was a blessing, considering marriage would consign her to being her husband’s property. She couldn’t be wed, now; none would want the Autumn prince’s leavings.

Turnabout was fair play, she supposed. Perhaps when he looked at her, he only saw the opportunity for revenge for Morrigan’s rebellion so long ago. He hadn’t kissed her like it was an act, though. He had kissed her like she was a wisp of smoke that would slide straight through his hands if he looked away. He had her backed against the wall, and still pressed against her like she might be wrenched from him.

It was Padraig who escorted her through the City and into the palace. Of course it would be a friend, a final slap as she was all but publicly sentenced. He avoided her eye.

Padraig left her in the hallway outside of Eris’ chambers, stepping back into the darkness without a word to her. Eris seemed to like her witty and clever, so Aisling tried to become that version of herself, to find that veneer again.

It disturbed her how easily she changed faces to suit who she spoke to. She turned herself into a mirror so that others could see their own desires reflected in her. She gave herself one breath more outside of his door. Just one last moment entirely to herself, to strengthen her resolve.

Nerves churned her stomach. Pass through this moment, she told herself. It is a means to an end. She raised her hand and knocked, and as she stepped through the doorway, the warmth of the room nearly bowled her back.

Fire – his fire – her eyes jumped to it immediately. Real flames, without wood. She knew this was his gift, the magic of Autumn, but she had never seen it. Wood had to be imported into the City and was too costly to burn casually. She rarely did so and had not seen a live fire in months.

“If you stare any harder, you’ll go blind.”

Aisling started at that, having forgotten for a fleeting moment why she was here, in Eris’ chambers. He sounded amused, but she found him hard to read and harder to know.

“We do not often have fires here,” she said, unsure of herself now that she was here. She felt hesitant and awkward, heat crawling up her neck. Her eyes flitted around the room, taking in the gilded carvings, the fine paintings – and the large bed. Her face felt hot.

“Touch the flames if you’d like,” Eris said, a hint of something dark in his voice. She stayed well away from the hearth.

“Thanatos arranged for this,” she said, gesturing vaguely. Her heart pounded in her chest, and she twisted a silver ring around her little finger. “I was given no choice. What was my agreed price?”

She forced herself to still, glancing at Eris shyly, before she abruptly realised that she didn’t need to look away. She wasn’t served by being shy, by playing coy. The males spoke of her as if she were property to trade between them, why should she feel the slightest hint of shame to look where she desired? Why did she deny herself even the barest hint of pleasure lest it be exploited? She was already exploited. That bridge had been crossed. They could do no worse to her than what had already been done. She looked upon Eris and drank her fill of him.

He lounged in the chair, the angles of his handsome face sharpened by the flickering light of his fire in the hearth. His expression was a picture of bored arrogance but belied by the intensity with which he studied her. Aisling’s skin prickled under the scrutiny of his gaze, feeling rather like a mouse before a cat. She felt the whisper of wards closing over the door, brushing past her like cobwebs, and she shivered.

“It was steep,” he said, meeting her eyes. “Be flattered. You were not traded cheaply.”

“I hope the cost was dear to you,” she said, irritation stabbing through her again at the way he spoke of it. Eris’ lips quirked up at her display of temper, no matter how mild; it surprised her. She had expected to be handled like chattel, like his reputation suggested.

“It was,” he admitted, no hint of shame in him. “For my part, I hope you are worth the price I paid.”

———————

This was torturous. Why had he agreed to this?

Aisling was stood still as a statue, looking every inch a noble daughter of Night in her shirt that bared her stomach and arms and gauzy, full skirt. Her face was blank and mild, as it had been every time he saw her bar last, but he knew she was boiling with irritation. Only her eyes, narrowed at him like an animal, gave her away.

Eris wanted her badly. He could admit it to himself. Ever since he had touched her that first time, he had wanted her.

“Why?” She asked, and he knew she meant all of it, not just the night stretching before them.

“Perhaps I’m soft-hearted and wanted to spare you from the nasty business of a forced marriage,” Eris couldn’t resist goading her, trying to see a glimpse of that temper again. She hid it well, and he wanted to peel back her every layer. “You asked me so nicely, after all.”

“Is being your consort a better fate?” She tilted her head.

“Yes,” he said smugly, and laughed at her responding huff. He beckoned her to him with two crooked fingers.

Eris was delighted when she obeyed. She glided to a stop directly before where he sat. So close he could touch her now, with none watching. He studied her, admiring the pale skin, the neckline that revealed the swell of her chest. The lushness of her, despite being reared in this hole. She wasn’t made to be kept down here.

“I won’t force you,” he finally said, when the tension grew long. “We can say whatever you like happened tonight. Only we need to know the truth. My decision is set, regardless of your choice.”

She tilted her head again, studying him, and he suddenly wondered what she thought of what she saw. Hesitantly, she reached out, fingers skimming over his shoulder. It took all his leashed self-control not to touch her in return, but to sit still, let her explore. If he wanted to get his cock in her – and he dearly wanted to – he had to give her an ounce of control.

“Will you hurt me?” She asked skittishly, her other hand coming to lightly stroke down his arm. Her fingertips ran to where his sleeves had been pushed to his elbow, and he hesitated before touching his bare skin. Eris laughed.

“No,” he said, and caught one of her hands with his. He brought it to his mouth, brushing his lips to her palm gently. Her arousal bloomed like a rose around him and he grew achingly hard. “Maybe you’ll scream, but I promise you’ll enjoy every second of it. Come here. Let me show you.”

He tugged her to him, pulling her down onto his lap. She hesitantly gripped his shoulders, stiff and awkward now, a little unsure. Eris gave in to his earlier urge and ran his hand over her bare waist, feeling the way she grew taut under his touch.

“Good,” he murmured, and she softened against him with the praise. His other hand came to hold the back of her head, and he tilted her, bringing his mouth to hers.

Eris kissed her languidly – he had all night – but the heat in it made him greedy. Aisling responded immediately, melting into him like he wanted. His hand skimmed up her waist, his thumb dipping under the hem of her shirt, stroking her gently. When the pad of his thumb brushed the underside of her breast she gasped, arching a bit against him, and he groaned at the friction against his cock.

“I think Lord Thanatos sent me here to punish me,” Aisling said as she pulled away, her voice breathy. Her fingers flexed into his shoulder as his hand skimmed down her back, tracing the knobs of her spine softly. How could so much magic be packed into such a delicate cage? He was surprised the force of it didn’t blow her ribs out, crush her lungs into dust. He forced himself to focus, despite the curve of her ass rubbing against his cock and demanding his full attention.

“I grievously wronged him when I was a child. He seemed eager for me to suffer at your hand,” she continued, and he heard the anxiety underpinning her desire.

“I’ve never harmed a female that’s come to my bed,” Eris said, his voice rough even to his ears, shifting her in his arms and grinning wolfishly. “I prefer my bedwarmers entirely eager and willing. Though if you say you don’t want me, I’ll say you’re a liar.”

“I thought you were the liar,” Aisling murmured, eyes closing as he slid his hand back under her top, the pad of his thumb brushing over her nipple. The breathy little gasp she made frayed any self-control he had remaining.

“Enough of that talk now,” he commanded. “Have you been touched before?”

He kept his hands moving over her, sweeping over her ribs and trailing down her back. It was meant to be both soothing and arousing, getting her used to being touched by him, gentling her to him.

“Yes,” she murmured after a moment, eyes still wide.

“Only by a male’s hands?” Eris asked, slowly pulling at the laces at the back of her top.

“By mouth as well,” she whispered, making Eris laugh. She was so typically high fae, stretching a word to its breaking point.

“And they said you were a maiden,” he teased. He pulled the last lace and her top fell away, leaving her breasts bare for his admiration. She was lovely, nervous and playful and flushed with desire all in one. He pressed his mouth against her throat, breathing in the scent of her, floral and dark. It was making him feel drunk, and he felt his mind fogging over as her pulse raced beneath his lips. She moaned in his lap as his hand came up to toy with her breast, hips canting against him, and it was pure instinct that made him bite.

Eris laughed again at her furious hiss, picking her up as she tried to squirm away. Laying her on the bed, he pulled at the lace to her skirt and neatly divested her of it, standing for a moment to admire her fully naked and splayed out before him. His cock throbbed painfully but he kept his trousers on still, knowing he had a task before him if he hoped to ever have her in his bed willingly again.

Aisling propped herself up on her elbows, studying him with those dark, fathomless eyes, watching as he gently lifted one of her legs to his shoulder. He pressed his mouth to the inside of her knee softly, hearing her breathing pick up as he kissed his way down until he was kneeling before her. The scent of her arousal flooded around him and he groaned at the sight of her, desperately wet. He pulled both of her legs over his shoulders, savouring the way she trembled before he’d even really begun. She let out a little gasp and his already frayed tether snapped.

“Wait -” she said, trying to squirm away, but he dragged her hips back towards him and licked a broad stripe up her centre. She keened underneath him as he traced his tongue over her, unable to resist plunging it inside her.

He had a reputation in Autumn for eating pussy, even in a Court of rowdy, lively lovers. He’d had perhaps more than his fair share of females over the centuries, his status as heir apparent making him a desirable catch. But he had never tasted anything like Aisling, sweet as honey and rich as red wine.

He spread her legs to better fit his shoulders between them, gripping her thighs as he flicked her clit with the tip of his tongue. Her hips tried to jump from the mattress, angling against his face; he risked a glance up and had to fist his cock through his trousers at the sight of her. Her back was arched, black hair tousled and wild, hands gripping the sheets desperately. What a fucking gift he’d been given.

That thought brought him back to task. He stroked her slit with his middle finger, slipping it in easily from how drenched she was. She balked when he worked in a second, so he sucked at her clit until she was pliant and moaning again, curling his fingers gently against her walls, working them in and out.

He crooked his fingers gently, curling them forward and making her back arch from the bed. Eris carefully built a steady pace, toying with her clit and kissing her thigh, until he gauged her ready from the sodden feel of her.

At the insistent press of a third finger she nearly shot off the bed. He threw his other arm over her hips, keeping her down.

“No -” she gasped, then cut off as he flicked her clit with his tongue again.

“Yes,” he said, near slurring. “Trust me. Relax.”

Aisling moaned but did as he bid, and as he slid his third finger into her impossibly tight cunt she bucked against his hand and panted. He pressed the flat of his tongue to her and crooked his fingers and she came with a near scream, so wet his fingers were dripping.

“So good,” he groaned as she caught her breath, panting wildly. “So fucking good.”

He had ripped off his shirt with one hand, kicking out of his boots and trousers, unable to wait a second fucking longer. He’d done what he could to prepare her, but as he moved over her the sudden reappearance of fear in her eyes made him get a hold of himself.

She brushed her hand over his hair tentatively, trailing her fingers over his cheekbones. He swore he meant to say something reassuring, something to ease her nerves.

“Beautiful,” he breathed instead, perhaps the softest thing he’d said to any female. He worked the tip of his cock into her at her look of surprise.

His eyes nearly rolled back in his head at the feel of her. Wet and soft and tight around his cock, like a silk glove tailored to fit him and him alone.

He had done a good job working her open. Her cunt didn’t offer much resistance as he sunk in slowly, slipping into the velvet plushness of her with a groan.

“Eris,” she whimpered underneath him, and the sound of his name said in that wanton way made him press his mouth to hers greedily. He rolled his hips softly and she moaned again, grabbing his shoulders. He began to move in earnest then, thrusting into her eagerly, one hand cupping her lovely face and the other toying with her clit. He pressed his forehead to hers, lost in the hot, slick clutch of her cunt, the unbelievable feel of her. He wanted to make her cum around his cock, he wanted to watch her fall apart again at his hands, he wanted to hear his name spill from her lovely lips again -

“Eris?!” Aisling said, and the sudden panic in her voice and her hands digging into his shoulders made his eyes snap open. Her eyes were wild with fright, entirely startled, and he managed to still himself in a monumental act of self-control. She had been enjoying herself a second ago, rolling her hips up to meet him like she’d taken his cock a hundred times before.

“Aisling,” he said, stilling and tracing her cheekbone with his thumb. “Wha-”

The bond snapped with all the force of a punch to the ribs.

[otterloreart]

the donut tutorial is too long so heres the abridged version

thought i would put my money where my mouth is and make my own donut tutorial. google blender donut tutorial if you want to see the original i dont need to give that guy any views lol.

left click new file

select box, press x, delete

*Shift*+a -> Mesh -> Torus

In the torus menu make the donut littler

left click the arrow at top right, check X ray box

left click this at top of screen. it will make a ring around your cursor when editing. see later on.

tab -> edit mode, numpad 1 to go to front view

double left click to select the middle line

ctrl + b to make more lines

double left click the middle line and hold S. scroll the mouses to adjust how much the nearby lines are changed

tab -> object mode, right click donut, left click Shade Smooth in menu

tab -> back to edit mode

used ctrl + b to add more lines at the top and bottom

left click and then use g to drag random vertices to shift parts of the donut and make it look authentically uneven (tm). scrolling while holding g makes the circles bigger and allows me to drag more vertices.

middle mouse button to rotate, shift + m button to move view to help see all around the donut

tab -> object mode, look at the beautiful donut

ok back to tab -> edit mode

numpad 1 for front view. drag left click to select the top half of the donut. shift + d to duplicate. right click to cancel move (when something is copied, "g" or "move" is automatically performed after that to move it)

press P, left click selection. you have removed those vertices from your object and made them a new object, see in viewer:

the duplicate object is named .001

tab -> object, select torus.001, tab -> edit.

click these buttons. this will make the vertices magnetically attach to the nearest object when you press g and move them.

left clicked the arrow at the far top right and hit random under color so i could see the difference between the frosting and donut

left click vertices and press g to pull them down to form a wave pattern around the donut. hold middle button to rotate and be sure to get all angles

just keep going

click 3 vertices in a row and drag them downwards to *e for extrude to* make a drip

each drip is about 3 extrudes long. g used to grab the vertices afterwards and shape the drip.

click the little wrench on the side for modifiers, torus.001 is listed at the top (aka frosting)

left click add modifier, click solidify

change the thickness to .02m (or whatever else looks good) and offset to +1.0000

left click add modifier, click subdivision surface

now things are smooth

apply solidify using this arrow. then apply subdivision.

tab -> sculpt mode

left click the lil inflate brush, uses on the tips of the drips to make them bulbous.

ok, to be continued.

[innovatexblog]

How Large Language Models (LLMs) are Transforming Data Cleaning in 2024

Data is the new oil, and just like crude oil, it needs refining before it can be utilized effectively. Data cleaning, a crucial part of data preprocessing, is one of the most time-consuming and tedious tasks in data analytics. With the advent of Artificial Intelligence, particularly Large Language Models (LLMs), the landscape of data cleaning has started to shift dramatically. This blog delves into how LLMs are revolutionizing data cleaning in 2024 and what this means for businesses and data scientists.

The Growing Importance of Data Cleaning

Data cleaning involves identifying and rectifying errors, missing values, outliers, duplicates, and inconsistencies within datasets to ensure that data is accurate and usable. This step can take up to 80% of a data scientist's time. Inaccurate data can lead to flawed analysis, costing businesses both time and money. Hence, automating the data cleaning process without compromising data quality is essential. This is where LLMs come into play.

What are Large Language Models (LLMs)?

LLMs, like OpenAI's GPT-4 and Google's BERT, are deep learning models that have been trained on vast amounts of text data. These models are capable of understanding and generating human-like text, answering complex queries, and even writing code. With millions (sometimes billions) of parameters, LLMs can capture context, semantics, and nuances from data, making them ideal candidates for tasks beyond text generation—such as data cleaning.

To see how LLMs are also transforming other domains, like Business Intelligence (BI) and Analytics, check out our blog How LLMs are Transforming Business Intelligence (BI) and Analytics.

Traditional Data Cleaning Methods vs. LLM-Driven Approaches

Traditionally, data cleaning has relied heavily on rule-based systems and manual intervention. Common methods include:

Handling missing values: Methods like mean imputation or simply removing rows with missing data are used.

Detecting outliers: Outliers are identified using statistical methods, such as standard deviation or the Interquartile Range (IQR).

Deduplication: Exact or fuzzy matching algorithms identify and remove duplicates in datasets.

However, these traditional approaches come with significant limitations. For instance, rule-based systems often fail when dealing with unstructured data or context-specific errors. They also require constant updates to account for new data patterns.

LLM-driven approaches offer a more dynamic, context-aware solution to these problems.

How LLMs are Transforming Data Cleaning

1. Understanding Contextual Data Anomalies

LLMs excel in natural language understanding, which allows them to detect context-specific anomalies that rule-based systems might overlook. For example, an LLM can be trained to recognize that “N/A” in a field might mean "Not Available" in some contexts and "Not Applicable" in others. This contextual awareness ensures that data anomalies are corrected more accurately.

2. Data Imputation Using Natural Language Understanding

Missing data is one of the most common issues in data cleaning. LLMs, thanks to their vast training on text data, can fill in missing data points intelligently. For example, if a dataset contains customer reviews with missing ratings, an LLM could predict the likely rating based on the review's sentiment and content.

A recent study conducted by researchers at MIT (2023) demonstrated that LLMs could improve imputation accuracy by up to 30% compared to traditional statistical methods. These models were trained to understand patterns in missing data and generate contextually accurate predictions, which proved to be especially useful in cases where human oversight was traditionally required.

3. Automating Deduplication and Data Normalization

LLMs can handle text-based duplication much more effectively than traditional fuzzy matching algorithms. Since these models understand the nuances of language, they can identify duplicate entries even when the text is not an exact match. For example, consider two entries: "Apple Inc." and "Apple Incorporated." Traditional algorithms might not catch this as a duplicate, but an LLM can easily detect that both refer to the same entity.

Similarly, data normalization—ensuring that data is formatted uniformly across a dataset—can be automated with LLMs. These models can normalize everything from addresses to company names based on their understanding of common patterns and formats.

4. Handling Unstructured Data

One of the greatest strengths of LLMs is their ability to work with unstructured data, which is often neglected in traditional data cleaning processes. While rule-based systems struggle to clean unstructured text, such as customer feedback or social media comments, LLMs excel in this domain. For instance, they can classify, summarize, and extract insights from large volumes of unstructured text, converting it into a more analyzable format.

For businesses dealing with social media data, LLMs can be used to clean and organize comments by detecting sentiment, identifying spam or irrelevant information, and removing outliers from the dataset. This is an area where LLMs offer significant advantages over traditional data cleaning methods.

For those interested in leveraging both LLMs and DevOps for data cleaning, see our blog Leveraging LLMs and DevOps for Effective Data Cleaning: A Modern Approach.

Real-World Applications

1. Healthcare Sector

Data quality in healthcare is critical for effective treatment, patient safety, and research. LLMs have proven useful in cleaning messy medical data such as patient records, diagnostic reports, and treatment plans. For example, the use of LLMs has enabled hospitals to automate the cleaning of Electronic Health Records (EHRs) by understanding the medical context of missing or inconsistent information.

2. Financial Services

Financial institutions deal with massive datasets, ranging from customer transactions to market data. In the past, cleaning this data required extensive manual work and rule-based algorithms that often missed nuances. LLMs can assist in identifying fraudulent transactions, cleaning duplicate financial records, and even predicting market movements by analyzing unstructured market reports or news articles.

3. E-commerce

In e-commerce, product listings often contain inconsistent data due to manual entry or differing data formats across platforms. LLMs are helping e-commerce giants like Amazon clean and standardize product data more efficiently by detecting duplicates and filling in missing information based on customer reviews or product descriptions.

Challenges and Limitations

While LLMs have shown significant potential in data cleaning, they are not without challenges.

Training Data Quality: The effectiveness of an LLM depends on the quality of the data it was trained on. Poorly trained models might perpetuate errors in data cleaning.

Resource-Intensive: LLMs require substantial computational resources to function, which can be a limitation for small to medium-sized enterprises.

Data Privacy: Since LLMs are often cloud-based, using them to clean sensitive datasets, such as financial or healthcare data, raises concerns about data privacy and security.

The Future of Data Cleaning with LLMs

The advancements in LLMs represent a paradigm shift in how data cleaning will be conducted moving forward. As these models become more efficient and accessible, businesses will increasingly rely on them to automate data preprocessing tasks. We can expect further improvements in imputation techniques, anomaly detection, and the handling of unstructured data, all driven by the power of LLMs.

By integrating LLMs into data pipelines, organizations can not only save time but also improve the accuracy and reliability of their data, resulting in more informed decision-making and enhanced business outcomes. As we move further into 2024, the role of LLMs in data cleaning is set to expand, making this an exciting space to watch.

Large Language Models are poised to revolutionize the field of data cleaning by automating and enhancing key processes. Their ability to understand context, handle unstructured data, and perform intelligent imputation offers a glimpse into the future of data preprocessing. While challenges remain, the potential benefits of LLMs in transforming data cleaning processes are undeniable, and businesses that harness this technology are likely to gain a competitive edge in the era of big data.

[naveenkrishna002]

Unveiling Market Insights: Exploring the Sampling Distribution, Standard Deviation, and Standard Error of NIFTY50 Volumes in Stock Analysis

Introduction:

In the dynamic realm of stock analysis, exploring the sampling distribution, standard deviation, and standard error of NIFTY50 volumes is significant. Providing useful tools for investors, these statistical insights go beyond abstraction. When there is market volatility, standard deviation directs risk evaluation. Forecasting accuracy is improved by the sample distribution, which functions similarly to a navigational aid. Reliability of estimates is guaranteed by standard error. These are not only stock-specific insights; they also impact portfolio construction and enable quick adjustments to market developments. A data-driven strategy powered by these statistical measurements enables investors to operate confidently and resiliently in the financial world, where choices are what determine success.           

NIFTY-50 is the tracker of Indian Economy, the index is frequently evaluated and re-equalizing to make sure it correctly affects the shifting aspects of the economic landscape in India. Extensively pursued index, this portrays an important role in accomplishing, investment approach ways and market analyses.

Methodology

The data was collected from Kaggle, with the (dimension of 2400+ rows and 8 rows, which are: date, open, close, high, low, volume, stock split, dividend. After retrieving data from the data source, we cleaned the null values and unnecessary columns from the set using Python Programming. We removed all the 0 values from the dataset and dropped all the columns which are less correlated.

After completing all the pre-processing techniques, we imported our cleaned values into RStudio for further analysis of our dataset.

Findings:

Our aim lies in finding how the samples are truly representing the volume. So, for acquiring our aim, we first took a set of samples of sizes 100 and 200 respectively. Then we performed some calculations separately on both of the samples for finding the mean, standard deviation, sampling distribution and standard error. At last we compared both of the samples and found that the mean and the standard deviation of the second sample which is having the size of 200 is more closely related to the volume.

From the above table, the mean of the sample-2 which has a size of 200 entity is 291642.5 and the mean of the sample-1 is 270745. From this result, it is clear that sample-2 is better representative of the volume as compared to sample-1

            Similarly, when we take a look at the standard error, sample-2 is lesser as compared to sample-1. Which means that the sample-2 is more likely to be closer to the volume.

Population Distribution.

As per the graph, In most of the days from the year 2017 to 2023 December volume of trading of NIFTY50 was between 1lakh- 2.8lakhs.

Sample Selection

We are taking 2 sample set having 100 and 200 of size respectively without replacement. Then we obtained mean, standard deviation and standard error of both of the samples.

Sampling Distribution of Sample- 1

From the above graph, the samples are mostly between 0 to 2 lakhs of volume. Also, the samples are less distributed throughout the population. The mean is 270745, standard deviation is 195270.5 and the standard error of sampling is 19527.01.

Sampling Distribution of Sample- 2

From the above graph, the samples are mostly between 0 to 2 lakhs of volume. Also, the samples are more distributed than the sample-1 throughout the volume. The mean is 291642.5, standard deviation is 186162.3 and the standard error of sampling is 13163.66.

Replication of Sample- 1

Here, we are duplicating the mean of every sample combination while taking into account every conceivable sample set from our volume. This suggests that the sample size is growing in this instance since the sample means follow the normal distribution according to the central limit theorem.

As per the above graph, it is clear that means of sample sets which we have replicated follows the normal distribution, from the graph the mean is around 3 lakhs which is approximately equals to our true volume mean 297456 which we have already calculated.

Conclusion

In the observed trading volume range of 2 lakhs to 3 lakhs, increasing the sample size led to a decrease in standard error. The sample mean converges to the true volume mean as sample size increases, according to this trend. Interestingly, the resulting sample distribution closely resembles the population when the sample mean is duplicated. The mean produced by this replication process is significantly more similar to the population mean, confirming the central limit theorem's validity in describing the real features of the trade volume.

[b2bitmedia]

Control Structured Data with Intelligent Archiving

Control Structured Data with Intelligent Archiving

You thought you had your data under control. Spreadsheets, databases, documents all neatly organized in folders and subfolders on the company server. Then the calls started coming in. Where are the 2015 sales figures for the Western region? Do we have the specs for the prototype from two years ago? What was the exact wording of that contract with the supplier who went out of business? Your neatly organized data has turned into a chaotic mess of fragmented information strewn across shared drives, email, file cabinets and the cloud. Before you drown in a sea of unstructured data, it’s time to consider an intelligent archiving solution. A system that can automatically organize, classify and retain your information so you can find what you need when you need it. Say goodbye to frantic searches and inefficiency and hello to the control and confidence of structured data.

The Need for Intelligent Archiving of Structured Data

You’ve got customer info, sales data, HR records – basically anything that can be neatly filed away into rows and columns. At first, it seemed so organized. Now, your databases are overloaded, queries are slow, and finding anything is like searching for a needle in a haystack. An intelligent archiving system can help you regain control of your structured data sprawl. It works by automatically analyzing your data to determine what’s most important to keep active and what can be safely archived. Say goodbye to rigid retention policies and manual data management. This smart system learns your data access patterns and adapts archiving plans accordingly. With less active data clogging up your production systems, queries will run faster, costs will decrease, and your data analysts can actually get work done without waiting hours for results. You’ll also reduce infrastructure demands and risks associated with oversized databases. Compliance and governance are also made easier. An intelligent archiving solution tracks all data movement, providing a clear chain of custody for any information that needs to be retained or deleted to meet regulations. Maybe it’s time to stop treading water and start sailing your data seas with an intelligent archiving solution. Your databases, data analysts and CFO will thank you. Smooth seas ahead, captain!

How Intelligent Archiving Improves Data Management

Intelligent archiving is like a meticulous assistant that helps tame your data chaos. How, you ask? Let’s explore:

Automated file organization

Intelligent archiving software automatically organizes your files into a logical folder structure so you don’t have to spend hours sorting through documents. It’s like having your own personal librarian categorize everything for easy retrieval later.

Efficient storage

This software compresses and deduplicates your data to free up storage space. Duplicate files hog valuable storage, so deduplication removes redundant copies and replaces them with pointers to a single master copy. Your storage costs decrease while data accessibility remains the same.

Compliance made simple

For companies in regulated industries, intelligent archiving simplifies compliance by automatically applying retention policies as data is ingested. There’s no danger of mistakenly deleting information subject to “legal hold” and avoiding potential fines or sanctions. Let the software handle the rules so you can avoid data jail.

Searchability

With intelligent archiving, your data is indexed and searchable, even archived data. You can quickly find that invoice from five years ago or the contract you signed last month. No more digging through piles of folders and boxes. Search and find — it’s that easy. In summary, intelligent archiving brings order to the chaos of your data through automated organization, optimization, compliance enforcement, and searchability. Tame the data beast once and for all!

Implementing an Effective Data Archiving Strategy

So you have a mind-boggling amount of data accumulating and you’re starting to feel like you’re drowning in a sea of unstructured information. Before you decide to throw in the towel, take a deep breath and consider implementing an intelligent archiving strategy.

Get Ruthless

Go through your data and purge anything that’s obsolete or irrelevant. Be brutally honest—if it’s not useful now or in the foreseeable future, delete it. Free up storage space and clear your mind by ditching the digital detritus.

Establish a Filing System

Come up with a logical taxonomy to categorize your data. Group similar types of info together for easy searching and access later on. If you have trouble classifying certain data points, you probably don’t need them. Toss ‘em!

Automate and Delegate

Use tools that can automatically archive data for you based on your taxonomy. Many solutions employ machine learning to categorize and file data accurately without human input. Let technology shoulder the burden so you can focus on more important tasks, like figuring out what to have for lunch.

Review and Refine

Revisit your archiving strategy regularly to make sure it’s still working for your needs. Make adjustments as required to optimize how data is organized and accessed. Get feedback from other users and incorporate their suggestions. An effective archiving approach is always a work in progress. With an intelligent data archiving solution in place, you’ll gain control over your information overload and find the freedom that comes from a decluttered digital space. Tame the data deluge and reclaim your sanity!

Conclusion

So there you have it. The future of data management and control through intelligent archiving is here. No longer do you have to grapple with endless spreadsheets, documents, files and manually track the relationships between them.With AI-powered archiving tools, your data is automatically organized, categorized and connected for you. All that structured data chaos becomes a thing of the past. Your time is freed up to focus on more meaningful work. The possibilities for data-driven insights and optimization seem endless. What are you waiting for? Take back control of your data and unleash its potential with intelligent archiving. The future is now, so hop to it! There’s a whole new world of data-driven opportunity out there waiting for you.    

[someidioticurl]

20 times in a row aside (and I wanna see each) I worry how long the time period for duplicate removal would be. 5 minutes? 5 days? 5 months? How many of the 'always reblog' posts do we have that we would keep on reblogging even if they showed up weekly?

Last thing I want is for a post to not show up cause someone else has reblogged it an hour ago. I wasn't online an hour ago!

FUCK YOU IF SOMEONE I FOLLOW WANTS TO REBLOG A POST 20 TIMES THEN I WANT TO SEE IT ON MY DASH 20 TIMES FUCK YOU TUMBLR FUCK YOU

[innovationalofficesolution]

Aligning BI Strategy with Microsoft’s Analytics Stack

In today’s data-driven world, aligning your Business Intelligence (BI) strategy with a robust analytics ecosystem is no longer optional—it’s essential. Microsoft’s analytics stack, centered around Power BI, Azure Synapse Analytics, and the broader Azure Data Services, offers a scalable, unified platform that can transform how organizations gather insights, make decisions, and achieve business goals.

For enterprises transitioning from Tableau to Power BI, integrating with Microsoft’s analytics stack is more than a technical shift—it’s a strategic opportunity.

Why Microsoft’s Analytics Stack?

Microsoft’s stack is designed with synergy in mind. Power BI serves as the front-end visualization tool, while Azure Synapse Analytics acts as the powerhouse for data integration, big data analytics, and real-time processing. Azure Data Factory, Azure Data Lake, and SQL Server complement the environment by enabling seamless data movement, storage, and management.

Aligning with this ecosystem empowers organizations to:

Unify data access and governance

Leverage native AI and machine learning

Streamline collaboration via Microsoft 365 integration

Improve performance with cloud-scale analytics

Key Considerations for BI Strategy Alignment

1. Define Strategic Goals Clearly Start with identifying what you want to achieve—whether it’s real-time reporting, predictive analytics, or better self-service BI. Microsoft’s platform offers the flexibility to scale BI initiatives based on maturity and business priorities.

2. Optimize Data Architecture Unlike Tableau’s more visual-centric architecture, Power BI thrives in a model-driven environment. Organizations should design dataflows and models to fully leverage Power BI’s DAX capabilities, semantic layers, and integration with Azure SQL and Synapse.

3. Leverage Azure Synapse for Enterprise-Scale Analytics Synapse enables unified analytics over big data and structured data. When aligned with Power BI, it removes data silos and allows for direct querying of large datasets, which enhances performance and reduces duplication.

4. Automate with Azure Data Factory A well-aligned BI strategy includes efficient ETL processes. Azure Data Factory helps automate pipelines and data transformations that feed clean data into Power BI for analysis, reducing manual effort and errors.

5. Prioritize Governance and Security With Microsoft Purview and Power BI's Row-Level Security (RLS), organizations can ensure data compliance and user-level control over access. This becomes increasingly vital during and after a migration from platforms like Tableau.

A Strategic Migration Opportunity

For those moving from Tableau to Power BI, aligning with Microsoft’s full analytics stack opens doors to advanced capabilities previously underutilized. Tools like Pulse Convert by OfficeSolution help automate and optimize this migration process, ensuring that your data assets, dashboards, and logic align smoothly with Power BI’s architecture.

Final Thoughts

Aligning your BI strategy with Microsoft’s analytics stack isn't just a move to a new tool—it’s an investment in a future-ready, scalable, and intelligent data ecosystem. Whether you're migrating from Tableau or building from scratch, OfficeSolution is here to guide you in leveraging the full potential of Microsoft's platform for long-term analytics success.

[ckgupta07]

Data Analyst Interview Questions: A Comprehensive Guide

Preparing for an interview as a Data Analyst is difficult, given the broad skills needed. Technical skill, business knowledge, and problem-solving abilities are assessed by interviewers in a variety of ways. This guide will assist you in grasping the kind of questions that will be asked and how to answer them.

By mohammed hassan on Pixabay

General Data Analyst Interview Questions

These questions help interviewers assess your understanding of the role and your basic approach to data analysis.

Can you describe what a Data Analyst does? A Data Analyst collects, processes, and analyzes data to help businesses make data-driven decisions and identify trends or patterns.

What are the key responsibilities of a Data Analyst? Responsibilities include data collection, data cleaning, exploratory data analysis, reporting insights, and collaborating with stakeholders.

What tools are you most familiar with? Say tools like Excel, SQL, Python, Tableau, Power BI, and describe how you have used them in past projects.

What types of data? Describe structured, semi-structured, and unstructured data using examples such as databases, JSON files, and pictures or videos.

Technical Data Analyst Interview Questions

Technical questions evaluate your tool knowledge, techniques, and your ability to manipulate and interpret data.

What is the difference between SQL's inner join and left join? The inner join gives only the common rows between tables, whereas a left join gives all rows of the left table as well as corresponding ones of the right.

How do you deal with missing data in a dataset? Methods are either removing rows, mean/median imputation, or forward-fill/backward-fill depending on context and proportion of missing data.

Can you describe normalization and why it's significant? Normalization minimizes data redundancy and enhances data integrity by structuring data effectively between relational tables.

What are some Python libraries that are frequently used for data analysis? Libraries consist of Pandas for data manipulation, NumPy for numerical computations, Matplotlib/Seaborn for data plotting, and SciPy for scientific computing.

How would you construct a query to discover duplicate values within a table? Use a GROUP BY clause with a HAVING COUNT(*) > 1 to find duplicate records according to one or more columns.

Behavioral and Situational Data Analyst Interview Questions

These assess your soft skills, work values, and how you deal with actual situations.

Describe an instance where you managed a challenging stakeholder. Describe how you actively listened, recognized their requirements, and provided insights that supported business objectives despite issues with communication.

Tell us about a project in which you needed to analyze large datasets. Describe how you broke the dataset down into manageable pieces, what tools you used, and what you learned from the analysis.

Read More....

[vastenigmapanther]

data cleansing

🧹 Common Data Cleansing Tasks ChatGPT Can Handle

ChatGPT is capable of assisting with various data cleaning operations, including:

Standardizing text formats: Converting text to a consistent case (e.g., all uppercase or lowercase).

Correcting inconsistent entries: Aligning variations of similar entries (e.g., "NY" vs. "New York").

Handling missing values: Identifying and filling in missing data points.

Removing duplicates: Detecting and eliminating duplicate records.

Parsing and formatting dates: Ensuring date fields follow a consistent format.

Flattening nested data structures: Transforming complex data into a flat, tabular format.

Validating data entries: Checking for and correcting invalid data entries.robertorocha.infoPackt

For instance, in a dataset containing employee information, ChatGPT can standardize inconsistent name formats and unify various date formats for joining dates. KDnuggets+2Packt+2robertorocha.info+2

🛠️ How to Use ChatGPT for Data Cleaning

To utilize ChatGPT for data cleaning, follow these steps:tirabassi.com

Prepare Your Dataset: Ensure your data is in a structured format, such as CSV or Excel.

Upload the File: In ChatGPT, click the "+" icon to upload your dataset.

Describe the Cleaning Task: Clearly specify what cleaning operations you want to perform. For example:https://data-finder.co.uk/service/data-cleansing/

"Please standardize the 'Employee Name' column to title case and convert the 'Joining Date' column to the YYYY-MM-DD format."

Review and Execute: ChatGPT will generate and execute Python code to perform the specified cleaning tasks, providing you with the cleaned dataset.Medium+2OpenAI+2Packt+2

This approach is particularly beneficial for users without extensive coding experience, as ChatGPT handles the scripting and execution of data cleaning operations. Medium+3OpenAI+3StatsAmerica+3

⚠️ Considerations and Limitations

While ChatGPT offers significant advantages in data cleaning, be mindful of the following:

Accuracy: ChatGPT performs best with well-defined tasks. Ambiguous instructions may lead to suboptimal results.

Data Sensitivity: Avoid uploading confidential or sensitive data, especially when using non-enterprise versions of ChatGPT.

Scalability: For very large datasets (e.g., millions of rows), consider breaking the data into smaller chunks or using specialized data processing tools. OpenAI Community

📚 Additional Resources

For a visual guide on using ChatGPT for data cleaning, you might find this tutorial helpful:

Feel free to share a sample of your dataset or describe your specific data cleaning

[advancedexcelinstitute]

Excel Power Query vs. Power Pivot: Which Tool Should You Use?

If you’ve ever worked with large datasets in Excel, you’ve probably reached a point where basic formulas just aren’t enough. That’s where two of Excel’s most powerful features come in: Power Query and Power Pivot. Both tools are essential for data analysis, but they serve different purposes.

So how do you decide which one to use? In this guide, we’ll walk through the differences between them and help you figure out the right tool for your needs.

What Is Power Query in Excel?

Power Query in Excel is a tool designed to clean, transform, and prepare data for analysis. It allows you to import data from different sources, fix formatting issues, and shape the data exactly how you want it, all without changing the original files.

Key Benefits of Power Query:

Connects to many sources: Excel files, text files, databases, online sources, and more.

Cleans data efficiently: You can remove duplicates, split columns, filter rows, and convert data types.

No need for coding: Its visual interface makes data prep easy, even for non-programmers.

Keeps a clear log: Every step is recorded, so changes are easy to trace or undo.

If you regularly work with messy data from different departments or systems, Power Query is the tool that helps you get it all in one place and ready to go.

What Is a Power Pivot? A Practical Tutorial

Power Pivot is an advanced data modeling feature in Excel. Instead of just working with a single flat table, it lets you work with multiple related tables. You can create relationships, use calculated fields, and build powerful pivot tables from huge datasets.

What Power Pivot Does Best:

Handles big data: It can work with millions of rows without slowing down.

Creates relationships: You can link multiple tables without merging them.

Supports DAX formulas: These allow you to create advanced calculations that go beyond standard Excel functions.

Improves reporting: Helps build dynamic dashboards and pivot tables based on complex models.

If you’re building reports that need to pull information from several tables, Power Pivot will save you hours of work and improve the quality of your analysis.

Power Query vs Power Pivot: A Side-by-Side Comparison

Let’s break it down clearly:FeaturePower Query in ExcelPower Pivot TutorialMain Use Preparing and transforming data  Modeling data and running advanced       calculationsStrength Connecting and cleaning data from   sources Creating relationships and custom   measuresInterface Step-by-step visual editor Data model view with DAX supportIdeal For Standardizing messy input data Analyzing large structured datasets

So, Power Query is for cleaning and prepping. Power Pivot is for modeling and analyzing. Simple as that.

Using Both Tools Together

You don’t have to choose between them. In fact, combining both tools gives you the best results.

A Typical Workflow:

Start with Power QueryLoad the data, clean it, and apply all your transformations.

Move to Power PivotCreate relationships between the tables, define measures, and build your reports.

This combination is what makes Excel a real powerhouse for data analysis. It’s a workflow many professionals use daily.

When to Use Power Query

Here are situations where Power Query is the better choice:

You need to import data from multiple sources.

You’re dealing with messy or inconsistent data.

You want to automate data cleanup tasks.

You need a repeatable process that updates with fresh data.

If you find yourself repeating the same steps every time you get new data, Power Query can handle all of that with just one click.

When Power Pivot Is the Right Tool

Use Power Pivot when:

Your data is too large for regular Excel to handle efficiently.

You’re working with multiple related tables.

You want to build custom KPIs and metrics.

You need to create a dynamic dashboard with slicers and filters.

Power Pivot is perfect for business analysts who need to dive deep into data and build powerful reports without leaving Excel.

Final Thoughts

Understanding how Power Query in Excel and Power Pivot work, And how they work together can completely change the way you use Excel. They are part of a bigger trend of self-service BI tools, giving more power to users without relying on IT or external software.

Use Power Query to clean and organize your data.

Use Power Pivot to model and analyze it.

Use both tools to build a streamlined, automated workflow that saves time and improves accuracy.

If you’re serious about improving your Excel skills, learning both tools is a smart investment.

[beyond-a-name]

While this is greatly appreciated that tumblr staff are now being so transparent, there's a lot of this that raises some concerns:

Presenting only seeing people you follow in the "following" tab as a problem. The greatest unique usefulness of Tumblr, indeed the thing that keeps 90% of established users here who facilitate the generation of "content", is the chronological feed. Putting new creators here or shifting to an algorithmic focus will harm your website. New users may not know how to use a non-algorithmic website, but that leads into the next point.

Refusal to teach new users how to use the site. Rather than actually showing or explaining how tumblr works on sign-up, this process has been shunted to the responsibility of established users. While tumblr does operate very differently from other websites, this is a strength, not a weakness. It is tumblr's fault if the majority of people don't know how to use it, but this can be solved by more clearly communicating the current process both explicitly (by directly telling the user on sign-up) and implicitly (by providing design elements that clarify and encourage this use of the platform).

Lack of focus on improving the search function on tumblr. If you want to ensure people able can easily find new content but that you don't intrude upon the ability to curate your own experience (which again, is the main reason most of us are still here), then simply letting people find what they are looking for would be the best way to do that. If a user does not want to see new discoveries and you force it upon them, you have damaged that user's relationship to the platform. Those additional roadblocks encountered by the user could be remedied if they could actually find what they were looking for.

Using coercive design to bring users back to tumblr. If you want to ensure that notifications are positively recieved and not spammy, simply let the user decide the rate and purpose of those notifications. Do not send someone emails when they have push notifications turned off; if they do not want to be notified of something and you send them an email anyway, that is the most literal definition of spam.

Potentially removing "duplicate reblogs" also harms the identity of tumblr. Currently, the reblog function conveniently acts as a way to show that you like something not just once, but that you like it several times over or that you really really like it. This is actually invaluable feedback to creators and followers, as well as a genuine source of humour within the browsing experience. When I'm having a bad time with tumblr, seeing my mutual reblog something 20 times in a row makes me happy I'm using tumblr.

Ultimately, while this transparency is appreciated, many of these proposed changes and focuses are downright antithetical to all of the value tumblr actually does hold. You can simply teach people how to use the site, explain things more clearly upon sign-up, and fix the existing functions (particularly the search function), on tumblr that exist to curate your own experience. In the opening, you said the quiet part loud, that you believe the focus on curating your own experience on tumblr is a problem. However, it is the only real strength of the platform. Those snappy conversations that you are eager to foster only exist because you can curate your own experience. By pushing algorithmic feeds on tumblr or damaging the ability to curate your own experience, users will lose the ability and the incentive to keep up with those conversations because they can no longer facilitate their own communities.

Making tumblr algorithm-driven and solely focused on the experience of hypothetical new users will ensure that all of those new users sign-up to an empty platform.

Tumblr’s Core Product Strategy

Here at Tumblr, we’ve been working hard on reorganizing how we work in a bid to gain more users. A larger user base means a more sustainable company, and means we get to stick around and do this thing with you all a bit longer. What follows is the strategy we're using to accomplish the goal of user growth. The @labs group has published a bit already, but this is bigger. We’re publishing it publicly for the first time, in an effort to work more transparently with all of you in the Tumblr community. This strategy provides guidance amid limited resources, allowing our teams to focus on specific key areas to ensure Tumblr’s future.

The Diagnosis

In order for Tumblr to grow, we need to fix the core experience that makes Tumblr a useful place for users. The underlying problem is that Tumblr is not easy to use. Historically, we have expected users to curate their feeds and lean into curating their experience. But this expectation introduces friction to the user experience and only serves a small portion of our audience. 

Tumblr’s competitive advantage lies in its unique content and vibrant communities. As the forerunner of internet culture, Tumblr encompasses a wide range of interests, such as entertainment, art, gaming, fandom, fashion, and music. People come to Tumblr to immerse themselves in this culture, making it essential for us to ensure a seamless connection between people and content. 

To guarantee Tumblr’s continued success, we’ve got to prioritize fostering that seamless connection between people and content. This involves attracting and retaining new users and creators, nurturing their growth, and encouraging frequent engagement with the platform.

Our Guiding Principles

To enhance Tumblr’s usability, we must address these core guiding principles.

Expand the ways new users can discover and sign up for Tumblr.

Provide high-quality content with every app launch.

Facilitate easier user participation in conversations.

Retain and grow our creator base.

Create patterns that encourage users to keep returning to Tumblr.

Improve the platform’s performance, stability, and quality.

Below is a deep dive into each of these principles.

Principle 1: Expand the ways new users can discover and sign up for Tumblr.

Tumblr has a “top of the funnel” issue in converting non-users into engaged logged-in users. We also have not invested in industry standard SEO practices to ensure a robust top of the funnel. The referral traffic that we do get from external sources is dispersed across different pages with inconsistent user experiences, which results in a missed opportunity to convert these users into regular Tumblr users. For example, users from search engines often land on pages within the blog network and blog view—where there isn’t much of a reason to sign up. 

We need to experiment with logged-out tumblr.com to ensure we are capturing the highest potential conversion rate for visitors into sign-ups and log-ins. We might want to explore showing the potential future user the full breadth of content that Tumblr has to offer on our logged-out pages. We want people to be able to easily understand the potential behind Tumblr without having to navigate multiple tabs and pages to figure it out. Our current logged-out explore page does very little to help users understand “what is Tumblr.” which is a missed opportunity to get people excited about joining the site.

Actions & Next Steps

Improving Tumblr’s search engine optimization (SEO) practices to be in line with industry standards.

Experiment with logged out tumblr.com to achieve the highest conversion rate for sign-ups and log-ins, explore ways for visitors to “get” Tumblr and entice them to sign up.

Principle 2: Provide high-quality content with every app launch.

We need to ensure the highest quality user experience by presenting fresh and relevant content tailored to the user’s diverse interests during each session. If the user has a bad content experience, the fault lies with the product.

The default position should always be that the user does not know how to navigate the application. Additionally, we need to ensure that when people search for content related to their interests, it is easily accessible without any confusing limitations or unexpected roadblocks in their journey.

Being a 15-year-old brand is tough because the brand carries the baggage of a person’s preconceived impressions of Tumblr. On average, a user only sees 25 posts per session, so the first 25 posts have to convey the value of Tumblr: it is a vibrant community with lots of untapped potential. We never want to leave the user believing that Tumblr is a place that is stale and not relevant. 

Actions & Next Steps

Deliver great content each time the app is opened.

Make it easier for users to understand where the vibrant communities on Tumblr are. 

Improve our algorithmic ranking capabilities across all feeds. 

Principle 3: Facilitate easier user participation in conversations.

Part of Tumblr’s charm lies in its capacity to showcase the evolution of conversations and the clever remarks found within reblog chains and replies. Engaging in these discussions should be enjoyable and effortless.

Unfortunately, the current way that conversations work on Tumblr across replies and reblogs is confusing for new users. The limitations around engaging with individual reblogs, replies only applying to the original post, and the inability to easily follow threaded conversations make it difficult for users to join the conversation.

Actions & Next Steps

Address the confusion within replies and reblogs.

Improve the conversational posting features around replies and reblogs. 

Allow engagements on individual replies and reblogs.

Make it easier for users to follow the various conversation paths within a reblog thread. 

Remove clutter in the conversation by collapsing reblog threads. 

Explore the feasibility of removing duplicate reblogs within a user’s Following feed. 

Principle 4: Retain and grow our creator base.

Creators are essential to the Tumblr community. However, we haven’t always had a consistent and coordinated effort around retaining, nurturing, and growing our creator base.  

Being a new creator on Tumblr can be intimidating, with a high likelihood of leaving or disappointment upon sharing creations without receiving engagement or feedback. We need to ensure that we have the expected creator tools and foster the rewarding feedback loops that keep creators around and enable them to thrive.

The lack of feedback stems from the outdated decision to only show content from followed blogs on the main dashboard feed (“Following”), perpetuating a cycle where popular blogs continue to gain more visibility at the expense of helping new creators. To address this, we need to prioritize supporting and nurturing the growth of new creators on the platform.

It is also imperative that creators, like everyone on Tumblr, feel safe and in control of their experience. Whether it be an ask from the community or engagement on a post, being successful on Tumblr should never feel like a punishing experience.

Actions & Next Steps

Get creators’ new content in front of people who are interested in it. 

Improve the feedback loop for creators, incentivizing them to continue posting.

Build mechanisms to protect creators from being spammed by notifications when they go viral.

Expand ways to co-create content, such as by adding the capability to embed Tumblr links in posts.

Principle 5: Create patterns that encourage users to keep returning to Tumblr.

Push notifications and emails are essential tools to increase user engagement, improve user retention, and facilitate content discovery. Our strategy of reaching out to you, the user, should be well-coordinated across product, commercial, and marketing teams.

Our messaging strategy needs to be personalized and adapt to a user’s shifting interests. Our messages should keep users in the know on the latest activity in their community, as well as keeping Tumblr top of mind as the place to go for witty takes and remixes of the latest shows and real-life events.  

Most importantly, our messages should be thoughtful and should never come across as spammy.  

Actions & Next Steps

Conduct an audit of our messaging strategy.

Address the issue of notifications getting too noisy; throttle, collapse or mute notifications where necessary.  

Identify opportunities for personalization within our email messages. 

Test what the right daily push notification limit is. 

Send emails when a user has push notifications switched off.

Principle 6: Performance, stability and quality.

The stability and performance of our mobile apps have declined. There is a large backlog of production issues, with more bugs created than resolved over the last 300 days. If this continues, roughly one new unresolved production issue will be created every two days. Apps and backend systems that work well and don't crash are the foundation of a great Tumblr experience. Improving performance, stability, and quality will help us achieve sustainable operations for Tumblr.

Improve performance and stability: deliver crash-free, responsive, and fast-loading apps on Android, iOS, and web.

Improve quality: deliver the highest quality Tumblr experience to our users. 

Move faster: provide APIs and services to unblock core product initiatives and launch new features coming out of Labs.

Conclusion

Our mission has always been to empower the world’s creators. We are wholly committed to ensuring Tumblr evolves in a way that supports our current users while improving areas that attract new creators, artists, and users. You deserve a digital home that works for you. You deserve the best tools and features to connect with your communities on a platform that prioritizes the easy discoverability of high-quality content. This is an invigorating time for Tumblr, and we couldn’t be more excited about our current strategy.

[greatonlinetrainingsposts]

How Do You Use a SAS Tutorial to Learn Data Cleaning Techniques?

Before you start analyzing data, it's important to understand how clean your dataset is. If your data has missing values, duplicate entries, or inconsistent formatting, it can throw off your entire analysis. Even the most advanced model won’t work well if the data going into it is flawed.

That’s where SAS programming comes in. When you follow a SAS tutorial, you’re not just learning how to write code—you’re learning how to think through data problems. A good tutorial explains what each step does and why it’s important.

Here’s how to use a SAS tutorial to build your data cleaning skills, step by step.

1. Start by Inspecting the Data

The first thing most SAS tutorials will show you is how to explore and inspect your dataset. This helps you understand what you’re working with.

You’ll learn how to use:

PROC CONTENTS to see the structure and metadata

PROC PRINT to view the raw data

PROC FREQ and PROC MEANS to check distributions and summaries

As you review the outputs, you’ll start spotting common problems like:

Too many missing values in key variables

Numbers stored as text

Values that don’t make sense or fall outside expected ranges

These early steps help you catch red flags before you go deeper.

2. Learn How to Handle Missing Data

Missing data is very common, and a good SAS tutorial will show you a few ways to deal with it.

This includes:

Using IF conditions to identify missing values

Replacing them with zeros, averages, or medians

Removing variables or rows if they’re not useful anymore

The tutorial might also explain when to fill in missing data and when to just leave it out. Real-world examples from healthcare, marketing, or finance help make the decisions easier to understand.

3. Standardize and Format Your Data

A lot of data comes in messy. For example, dates might be stored in different formats or categories might use inconsistent labels like "M", "Male", and "male".

With SAS programming, you can clean this up by:

Converting dates using INPUT and PUT functions

Making text consistent with UPCASE or LOWCASE

Recoding values into standardized categories

Getting your formatting right helps make sure your data is grouped and analyzed correctly.

4. Remove Duplicate Records

Duplicate records can mess up your summaries and analysis. SAS tutorials usually explain how to find and remove duplicates using:

PROC SORT with the NODUPKEY option

BY group logic to keep the most recent or most relevant entry

Once you understand the concept in a tutorial, you’ll be able to apply it to more complex datasets with confidence.

5. Identify Outliers and Inconsistencies

Advanced tutorials often go beyond basic cleaning and help you detect outliers—data points that are far from the rest.

You’ll learn techniques like:

Plotting your data with PROC SGPLOT

Using PROC UNIVARIATE to spot unusual values

Writing logic to flag or filter out problem records

SAS makes this process easier, especially when dealing with large datasets.

6. Validate Your Cleaning Process

Cleaning your data isn’t complete until you check your work. Tutorials often show how to:

Re-run summary procedures like PROC MEANS or PROC FREQ

Compare row counts before and after cleaning

Save versions of your dataset along the way so nothing gets lost

This step helps prevent mistakes and makes sure your clean dataset is ready for analysis.

Why SAS Programming Helps You Learn Faster

SAS is great for learning data cleaning because:

The syntax is simple and easy to understand

The procedures are powerful and built-in

The SAS community is active and supportive

Whether you're a beginner or trying to improve your skills, SAS tutorials offer a strong, step-by-step path to learning how to clean data properly.

Final Thoughts

Learning data cleaning through a SAS tutorial doesn’t just teach you code—it trains you to think like a data analyst. As you go through each lesson, try applying the same steps to a dataset you’re working with. The more hands-on experience you get, the more confident you’ll be.

If you want to improve your data analysis and make better decisions, start by getting your data clean. And using SAS programming to do it? That’s a smart first move.

[krupa192]

Mastering NumPy Broadcasting for Efficient Computation 

If you're working with Python for data science, you've probably come across NumPy, a powerful library for handling numerical data. One of NumPy’s standout features is broadcasting, which simplifies operations on arrays of different shapes without requiring manual adjustments. This not only enhances computational efficiency but also improves memory management, making it a must-know technique for data scientists and machine learning professionals. 

In this guide, we’ll break down NumPy broadcasting, explaining how it works and why it’s a game-changer for high-performance computing. We’ll also explore real-world applications and discuss how you can master these skills through the Online Data Science Course UAE. 

Why Does NumPy Broadcasting Matter? 

When working with large datasets, efficiency is crucial. Traditional element-wise operations require arrays to have the same dimensions, which can lead to increased memory usage and slower execution times. Broadcasting eliminates this limitation by allowing NumPy to automatically adjust smaller arrays, ensuring they align with larger ones without duplicating data. 

Key Advantages of Broadcasting: 

Faster computations: Eliminates the need for explicit looping. 

Optimized memory usage: Avoids unnecessary copies of data. 

Simplifies code: Enhances readability by removing manual reshaping. 

Understanding How NumPy Broadcasting Works 

To apply broadcasting, NumPy follows a set of rules when performing operations on arrays of different shapes: 

If the arrays have different dimensions, NumPy expands the smaller array by adding singleton dimensions (size 1) from the left until both arrays have the same number of dimensions. 

If dimensions differ, those with size 1 are stretched to match the corresponding dimension of the larger array. 

If the arrays are still incompatible, a ValueError is raised. 

Example 1: Adding a Scalar to an Array 

import numpy as np    matrix = np.array([[1, 2, 3], [4, 5, 6]])  # Shape (2,3)  scalar = 10  # Shape ()    result = matrix + scalar  print(result) 

Output:  [[11 12 13]  [14 15 16]] 

Here, the scalar is automatically expanded to match the shape of the array, enabling efficient element-wise addition. 

Example 2: Broadcasting a 1D Array to a 2D Array 

matrix_2d = np.array([[1, 2, 3], [4, 5, 6]])  # Shape (2,3)  vector = np.array([10, 20, 30])  # Shape (3,)    result = matrix_2d + vector  print(result) 

Output:  [[11 22 33]  [14 25 36]] 

NumPy expands the 1D array across rows to match the (2,3) shape, allowing seamless element-wise operations. 

Example 3: Multi-Dimensional Broadcasting 

array_3d = np.array([[[1], [2], [3]]])  # Shape (1,3,1)  array_2d = np.array([[10, 20, 30]])  # Shape (1,3)    result = array_3d + array_2d  print(result) 

Output:  [[[11 21 31]    [12 22 32]    [13 23 33]]] 

NumPy stretches the shapes to align properly and executes the addition efficiently. 

Real-World Applications of NumPy Broadcasting 

1. Speeding Up Machine Learning Workflows 

Broadcasting is heavily used in data normalization for training machine learning models. Instead of manually reshaping arrays, NumPy allows quick transformations: 

data = np.array([[50, 60, 70], [80, 90, 100]])  mean = np.mean(data, axis=0)  norm_data = (data - mean) / np.std(data, axis=0) 

This efficiently normalizes the dataset without unnecessary loops. 

2. Image Processing 

Broadcasting is widely applied in image manipulation, such as adjusting brightness levels across RGB channels: 

image = np.random.rand(256, 256, 3)  # A 256x256 RGB image  brightness = np.array([1.2, 1.1, 0.9])  adjusted_image = image * brightness 

Each colour channel is scaled independently, improving computational efficiency. 

3. Financial & Statistical Analysis 

In financial modeling, broadcasting simplifies calculations like percentage change computations: 

prices = np.array([100, 102, 105, 110])  returns = (prices[1:] - prices[:-1]) / prices[:-1] * 100 

This eliminates manual looping, making stock price analysis faster and more efficient. 

Master Data Science with Boston Institute of Analytics (BIA) in UAE 

If you're looking to enhance your expertise in data science, AI, and machine learning, mastering NumPy broadcasting is a crucial step. The Boston Institute of Analytics (BIA) offers a comprehensive Online Data Science Course UAE, covering: 

Python Programming & NumPy Fundamentals 

Advanced Machine Learning & AI Techniques 

Data Visualization & Statistical Analysis 

Big Data & Cloud Computing 

Why Choose BIA? 

Learn from Industry Experts: Gain insights from experienced data scientists. 

Hands-On Projects: Work on real-world datasets for practical learning. 

Globally Recognized Certification: Earn a professional credential to boost your career. 

Flexible Online Format: Learn at your own pace, from anywhere in the UAE. 

By enrolling in BIA’s Online Data Science Course, you’ll build a strong foundation in Python, NumPy, and advanced analytics techniques, preparing yourself for high-paying roles in data science. 

Final Thoughts 

NumPy broadcasting is a game-changer for anyone dealing with numerical computations. Whether you're working on machine learning models, image processing tasks, or financial data analysis, understanding broadcasting will help you write more efficient and scalable code. 

Ready to take your data science journey to the next level? Join the Data Science Course today and gain industry-relevant skills that will set you apart in the competitive job market! 

[globosetechnologysolutionspvt]

How to Clean and Preprocess AI Data Sets for Better Results

Introduction

Artificial Intelligence Dataset (AI) models depend on high-quality data to produce accurate and dependable outcomes. Nevertheless, raw data frequently contains inconsistencies, errors, and extraneous information, which can adversely affect model performance. Effective data cleaning and preprocessing are critical steps to improve the quality of AI datasets, thereby ensuring optimal training and informed decision-making.

The Importance of Data Cleaning and Preprocessing

The quality of data has a direct impact on the effectiveness of AI and machine learning models. Inadequately processed data can result in inaccurate predictions, biased results, and ineffective model training. By adopting systematic data cleaning and preprocessing techniques, organizations can enhance model accuracy, minimize errors, and improve overall AI performance.

Procedures for Cleaning and Preprocessing AI Datasets

1. Data Collection and Analysis

Prior to cleaning, it is essential to comprehend the source and structure of your data. Identify key attributes, missing values, and any potential biases present in the dataset.

2. Addressing Missing Data

Missing values can hinder model learning. Common approaches to manage them include:

Deletion: Removing rows or columns with a significant number of missing values.

Imputation: Filling in missing values using methods such as mean, median, mode, or predictive modeling.

Interpolation: Estimating missing values based on existing trends within the dataset.

3. Eliminating Duplicates and Irrelevant Data

Duplicate entries can distort AI training outcomes. It is important to identify and remove duplicate records to preserve data integrity. Furthermore, eliminate irrelevant or redundant features that do not enhance the model’s performance.

4. Managing Outliers and Noisy Data

Outliers can negatively impact model predictions. Employ methods such as

The Z-score or Interquartile Range (IQR) approach to identify and eliminate extreme values.

Smoothing techniques, such as moving averages, to mitigate noise.

5. Data Standardization and Normalization

To maintain uniformity across features, implement:

Standardization: Adjusting data to achieve a mean of zero and a variance of one.

Normalization: Scaling values to a specified range (e.g., 0 to 1) to enhance model convergence.

6. Encoding Categorical Variables

Machine learning models perform optimally with numerical data. Transform categorical variables through:

One-hot encoding for nominal categories.

Label encoding for ordinal categories.

7. Feature Selection and Engineering

Minimizing the number of features can enhance model performance. Utilize techniques such as:

Principal Component Analysis (PCA) for reducing dimensionality.

Feature engineering to develop significant new features from existing data.

8. Data Partitioning for Training and Testing

Effective data partitioning is essential for an unbiased assessment of model performance. Typical partitioning strategies include:

An 80-20 split, allocating 80% of the data for training purposes and 20% for testing.

Utilizing cross-validation techniques to enhance the model's ability to generalize.

Tools for Data Cleaning and Preprocessing

A variety of tools are available to facilitate data cleaning, such as:

Pandas and NumPy, which are useful for managing missing data and performing transformations.

Scikit-learn, which offers preprocessing methods like normalization and encoding.

OpenCV, specifically for improving image datasets.

Tensor Flow and Pytorch, which assist in preparing datasets for deep learning applications.

Conclusion

The processes of cleaning and preprocessing AI datasets are vital for achieving model accuracy and operational efficiency. By adhering to best practices such as addressing missing values, eliminating duplicates, normalizing data, and selecting pertinent features, organizations can significantly improve AI performance and minimize biases. Utilizing sophisticated data cleaning tools can further streamline these efforts, resulting in more effective and dependable AI models. 

For professional AI dataset solutions, visit Globose Technology Solutions to enhance your machine learning initiatives.