codeworkaround

[codeworkaround]

Any software is not complete unless it sends an email.

Any dashboard is not complete unless it has a pie chart.

Any LLM model is not complete unless it generates and executes a SQL query.

[codeworkaround]

Back-of-the-Envelope

The problem with Back-of-the-Envelope solutions is that it leads smart and proud engineers to spend months defending a wrong conclusion that they only researched for 15 minutes.

[codeworkaround]

Evil ++i

i-=-1

[codeworkaround]

The state of interesting/cool things in software development

It is rather difficult to keep up to date with software development, things come and go, as they do in any industry, but the churn rate for tech is just ridiculous. In a matter of months, the industry trend shifts.

I try to keep up by reading and code tinkering but trying to guess what will stand the test of time is rather difficult. So let’s collect the interesting things at the beginning of 2017 and maybe review them after a year. It does not necessarily mean they are new/newest or unproven.

Docker

Is seems that everything is about containers now a days. Docker tries to solve the “it works on my machine” class of problems by wrapping up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries. Seems nice, solves the freelancing/small project deployment in the cloud. This one looks like it is here to stay after the novelty element runs out, I see a lot more people using it quite happy.

D3

I need charts. I love charts. A picture is worth a thousand words. On a browser via SVG from JavaScript? Yes I do, sign me up. It is data driven and great for interaction. Supports transitions. Also, has a great gallery for inspiration.

https://github.com/d3/d3/wiki/Gallery

In memory databases

I like SQL, not to the extent of returning Html out of it, but above average adopter. SQL does not get the love it deserves and I will never figure it out why we do not filter, sort, aggregate and verify constraints directly in the data layer. Every now and then, I see complete data sets up the application layers and it kills me. However, I digress.

Face it - we are gathering more and more data that needs to be analysed. Manage to get it in a structured form and stored in a database system that aggregates/filters on the fly via a robust and not so error prone language like SQL and you are half way there. At the current state of hardware, storing the data in-memory is the only to go - not the disk. I also love the fact that now a day you can get more than just aggregation with embedded libraries, being able to do regressions, clustering or time series analysis on billions of rows without going to R. It allows you to do more, to concentrate on data analysis, to do feature selection and not parallel batch processing.

.Net core

The dot net framework, but just as an open source platform, cross platform and trim down version. Uses NuGet, compatible with Docker, builds libraries, console apps and ASP.NET Core. In addition, you get the Visual Studio experience.

Honourable mention: Tensor Flow.

[codeworkaround]

The Secret Rules of Modern Living: Algorithms.

I really enjoyed Marcus du Sautoy's documentary The Secret Rules of Modern Living: Algorithms. I think it's entertaining even for the not so geeky inclined. Plus it gives you the opportunity to explain what you do at work, other then aligning a div vertically. Found it on Netflix. http://m.imdb.com/title/tt5818010/

[codeworkaround]

Ninety-ninety rule

It always confuses people when they actual add the time. But this is so true.

"The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time."

—Tom Cargill

[codeworkaround]

Links #12

How Google Cracked House Number Identification in Street View http://www.technologyreview.com/view/523326/how-google-cracked-house-number-identification-in-street-view/

The architecture of StackOverflow http://www.dev-metal.com/architecture-stackoverflow/

A Software Developer’s Reading List http://stevewedig.com/2014/02/03/software-developers-reading-list/

Code reading http://www.gigamonkeys.com/code-reading/

[codeworkaround]

Links #11

Andrew Tanenbaum: Writing the Book on Networks http://www.youtube.com/watch?v=i69Ngf8rips

Conversation with scientist, engineer and database legend Jim Gray http://channel9.msdn.com/shows/Behind+The+Code/Conversation-with-scientist-engineer-and-database-legend-Jim-Gray/

Strata NYC in Review: Enterprise Data Hubbub http://hortonworks.com/blog/strata-nyc-in-review-enterprise-data-hubbub/

[codeworkaround]

Tales of software development

This is my humble effort of summarizing hours upon hours of endless discussions on software development.

0. Reading list.

Read. Learn from those before you. If you will ever be involved in a software project I assure you that one of the following books will come in handy:

Frederick P. Brooks, The Mythical Man-Month (2nd edition), Addison-Wesley 1995

Steve McConell: Code Complete 2, Microsoft Press, 2004

Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides: Design Patterns: Elements of Reusable Object-Oriented Software (Addison-Wesley Professional Computing Series), Addison-Wesley 1995

A Hunt, D. Thomas: The Pragmatic Programmer: From Journeyman to Master, Addison-Wesley, 1994

And stop and read now Peter Norvig - Teach Yourself Programming in Ten Years : http://norvig.com/21-days.html

Don’t underestimate the experience of others. It takes at least 10,000 hours of practice to master any craft. Don’t think those hours of practice mastering this craft can ever be replaced. Respect those dedicated enough to put in those numbers.

1. Software development is a craft

Remember, the purpose of software development is actual software. Software development is not for everyone. If you think developers are interchangeable you are sorely mistaken. It’s like writing a novel and changing the writer in the middle of a sentence. It doesn’t work or if it works then you are writing a very, very bad novel.

There is a reason why Linust Torvals is still answering mailing lists for Linux kernel (http://www.wired.com/wiredenterprise/2013/07/linus-torvalds-right-to-offend/), why Anders Hejlsberg proved successful twice (with Delphi and C#) in a field where almost everyone else fails.

Practice. Code. Even if you are not actively coding do it sometimes in your own free time. Build products and when possible, use them. Never underestimate the power of eating your own dog food. http://en.wikipedia.org/wiki/Eating_your_own_dog_food

Lead by example. The best developers/architects/designers/product managers output  their work into successful products. You need all of them to build great products. Seek those reminiscing about great products. Do that yourself and document problems and solutions; never remember excuses. In the end, they are meaningless.

Trust the people you are working with. Assume that they are doing their best and don’t second guess their expertise. Let them have their choice of tools, algorithms, methodology, programing languages or databases. Many of them are almost religious about their choice.

2. Progress tracking

"Nine women can't make a baby in one month". - Fred Brooks.

Never use lines of code as a metric as you would never judge a book by it’s length. Always assign weights to functionality and bugs, remember to quantify the effort involved both for development and testing.

Remember that an estimate is not a promise and don’t think that throwing another developer (or resource) will complete the work faster. It almost never works. Coding/programing/development is all about the context, so always plan accordingly, allocate time for them to understand the business problem, the architecture, the constraints. Let people work on one task at a time, respect the concentration of a programmer submersed into a project and don’t break it, further estimates/reports/chit-chat are not more important than actual work. Context switching for humans comes with a great time penalty. So does CPU task switching but that is another topic. Any meeting, progress report, estimate is not as important as actual code.

3. Testing

The one and only purpose of testing is to check that the software runs as expected. Testing should be easy to re-run, regardless if it is unit testing, integration testing and system testing. Keep logs of tests performed over time but don’t overdo it with proofs of tests. Don’t waste anyone's time with screen-shots and reports. No one is paying for excuses.

Keep in mind that acceptance testing will trigger user feedback and new features. Be aware, this does mean failed testing, future development is to be expected on every software project but focus on delivering what you have promised in the beginning.

4. Customer/ user feedback

The customer/user has problems and needs. Listen very carefully and focus on the problem, not the solution he is suggesting. Most of them will state solutions to their problems, not the actual problem. Assume the role of a doctor, listen to the problem, the patient knows exactly what is bothering him but don’t prescribe a remedy based on patient recommendation. Self medicating is not advisable.

Build mock-ups, preferably live mock-ups, include real data, make the extra effort of A/B testing and sometimes invest in mock-up features that will never make it to production. Get feedback and be flexible in the beginning in accommodating change, but rigid in the later stages.

[codeworkaround]

Links #10

The dying platform: .Net http://roundcrisis.com/2013/05/15/the-dying-platform-net/

Where Is .NET Headed? http://odetocode.com/blogs/scott/archive/2013/05/15/where-is-net-headed.aspx

New course: Ruby on Rails - A Jumpstart for .NET Developers http://blog.pluralsight.com/2013/05/14/new-course-ruby-on-rails-a-jumpstart-for-net-developers/

C# To Objective-C - The Ultimate Guide http://www.infragistics.com/community/blogs/stevez/archive/2013/05/09/c-to-objective-c-the-ultimate-guide.aspx

April 2013 Release of the Ajax Control Toolkit http://stephenwalther.com/archive/2013/04/30/april-2013-release-of-the-ajax-control-toolkit.aspx

[codeworkaround]

Links #9

Stackoverflow.com: the road to SSL http://nickcraver.com/blog/2013/04/23/stackoverflow-com-the-road-to-ssl/

Categories, What's the Point? http://jeremykun.com/2013/04/16/categories-whats-the-point/

Typical Big Data Architecture http://venublog.com/2012/11/30/typical-big-data-architecture/

20 Free UI Templates for Sketch.app http://speckyboy.com/2013/04/22/free-ui-templates-for-sketch-app/

[codeworkaround]

Links #8

50 Amazing jQuery Plugins That You Should Start Using Right Now http://tutorialzine.com/2013/04/50-amazing-jquery-plugins/

Iconic Albuquerque Photo Re-Created http://tutorialzine.com/2013/04/50-amazing-jquery-plugins

http://www.microsoft.com/en-us/news/features/2008/jun08/06-25iconic.aspx

15 Programming Skills Most Coveted By Employers http://readwrite.com/2013/04/09/15-programming-skills-most-coveted-by-employers

[codeworkaround]

Links #7

Big Data & Data science http://www.slideshare.net/slideshow/embed_code/16190193#

A Practical Intro to Data Science http://blog.zipfianacademy.com/post/46864003608/a-practical-intro-to-data-science

The NBA Loves Stats. That’s a Problem http://www.slate.com/articles/sports/sports_nut/2013/02/nba_stats_gurus_can_t_work_together_anymore_that_s_a_problem.single.html

Principles of Software Engineering, Part 1 - thoughts from the red planet - thoughts from the red planet http://nathanmarz.com/blog/principles-of-software-engineering-part-1.html

[codeworkaround]

Links #6

Celebrating 100 Years of Markov Chains http://computefest.seas.harvard.edu/markov

Interview with Nick Chamandy, statistician at Google http://simplystatistics.org/2013/02/15/interview-with-nick-chamandy-statistician-at-google/

Performance-Wise, C# Trumps Java http://www.datacenteracceleration.com/author.asp?section_id=2933

7 Things Developers Should Know About SQL Server http://www.brentozar.com/archive/2013/02/7-things-developers-should-know-about-sql-server/

10 R packages I wish I knew about earlier http://blog.yhathq.com/posts/10-R-packages-I-wish-I-knew-about-earlier.html

[codeworkaround]

Links #5

Narrative reports vs dashboards http://strata.oreilly.com/2013/01/narrative-reports-vs-dashboards.html

A new development era (essay) http://blog.ziade.org/2013/01/25/a-new-development-era-essay/

Hadoop + SQL Server + Excel = Big Data Analytics http://www.infoq.com/news/2013/01/big-data-microsoft

Machine Learning Cheat Sheet (for scikit-learn) http://peekaboo-vision.blogspot.de/2013/01/machine-learning-cheat-sheet-for-scikit.html

100 most read R posts for 2012 (stats from R-bloggers) – big data, visualization, data manipulation, and other languages http://www.r-bloggers.com/100-most-read-r-posts-for-2012-stats-from-r-bloggers-big-data-visualization-data-manipulation-and-other-languages/

[codeworkaround]

Links #4

An Interactive Guide To The Fourier Transform http://betterexplained.com/articles/an-interactive-guide-to-the-fourier-transform/

SQL Advent 2012 Day 11: Cursors and loops http://blogs.lessthandot.com/index.php/DataMgmt/DBProgramming/MSSQLServer/cursors-and-loops

Setting up Source Control http://blogs.lessthandot.com/index.php/DataMgmt/DBAdmin/database-projects-setting-up-source

The Roslyn Incident http://neildanson.wordpress.com/2012/12/24/the-roslyn-incident/

Douglas Crockford: “Java was a colossal failure...JavaScript is succeeding because it works.” http://jaxenter.com/douglas-crockford-java-was-a-colossal-failure-javascript-is-succeeding-because-it-works-45928.html

[codeworkaround]

Executing(calling) a MS SQL stored procedure from R and retrieve the dataset

First thing is to establish a ODBC connection with SQL Server: Control Panel -> Administrative Tools -> ODBC Data Sources -> User DNS -> Add SQL Server Native Client 11.0. Then setting it up is straightforward. I recommend creating a SQL Server user just for this ODBC connection.

Next you need to download the RODBC package from CRAN - http://cran.r-project.org/web/packages/RODBC/index.html - and later install it from the package archive.

To call a MS SQL server stored procedure from R use the following code:

exec.store.proc <- function(param.name){ # load the RODBC package require("RODBC"); # establish a connection ch <- odbcConnect("RDB",uid="***",pwd="***"); # construct a query string with a parameter query <- paste("exec dbo.R_getData '",param.name, "'",sep=""); # execute the query df<-sqlQuery(ch, query); # print the results print(df); }

If the result of the following code is not the expected one and you receive:

> exec.store.proc('')

character(0)

please keep in mind that you need to use

set nocount on

in the body of the stored procedure otherwise the character(0) is frustrating.

if object_id('R_getData') is not null drop proc R_getData go create proc R_getData @name varchar(255)='' as --important, otherwise R will retun character(0) set nocount on select @name = upper(@name)+'%' create table #TMP (ID int,Name varchar(255), TypeDesc varchar(255),CreateDate datetime) insert into #TMP select [object_id],name, type_desc,create_date from sys.objects a where upper(name) like @name order by [object_id] select * from #TMP go