kalbarczyk

[kalbarczyk]

Why you shouldn’t use persistence automators in Redux

In mid-2018 I made an informed decision to move away from React Native in favor of Flutter. To finish my journey in a good style I decided to open source a template that I worked on for over two years. I also decided to write a few articles about architecture choices that I had to make throughout the years.

Preserving state is one of the most important parts of app architecture. The state keeps information about all things that define what the user sees and how they can interact with our app. The state is a big topic in web apps because of how browsers handle data, cookies (GDPR anyone?), backward compatibility, etc.

Historically the place to preserve state was a database installed on the backend. The reason for using databases was partially due to limitations of web browsers, partially lack of computation speed on machines that were running apps.

The misunderstanding of the concept of preserving state was so big that now all EU citizens have to click *accept cookies* to allow such fundamental feature as session data storage on the server.

In case of mobile app development, preserving state does the very same thing as the state on a web app (or game, or any other software). It keeps information so that the user doesn't need to configure the app and then click through the UI every time they open it.

The big difference here is that mobile platforms are much more capable in terms of access to the internals of the hardware than web browsers are. React Native app can access the file system, internal databases, networks, many API's at once, etc. Ways to preserve state are almost limitless.

And here is the issue with redux-persist. This library was intended for web browser usage and then adapted to work with React Native.

While redux-persist makes a lot of sense in web browser realm where the only way to save data locally is LocalStorage (which is very limited in functionality) and there are literally thousands of other restrictions including ephemeral nature of websites (you close the tab and the app is basically gone), it makes perfect sense to add to your project a layer that abstracts things for you. Yet on mobile, those restrictions don't exist and the disadvantages of react-persist are exposed in a very unpleasant way:

Performance

React Native has one JS thread. It means that if any function uses the processor, UI stops responding until the function finishes.

Redux store is one big JSON tree. If we want to serialize this tree to AsyncStorage, a resource-hungry JSON.stringify() and JSON.parse() is ran on each change of any value in the store.

I usually tackled this issue by storing only the most important things in AsyncStorage and dividing data into many keys. This was efficient only when the persistence layer was decoupled from the store, which is very hard if not impossible in redux-persist.

To tackle performance issues in redux-persist, we need to use whitelisting, blacklisting, and filtering. But even with those niceties, we still use CPU on each store update and not necessary when we really want to - it's still automatic.

Maintaining persistence rules is very hard and error-prone.

Lack of integration with other libraries

Most notable is Immutable.JS. If you use Immutable.JS with Redux, it's best when the whole state is an Immutable Map. This makes working with business logic like a charm. Redux-preserve v4 had a wrapper that made it work with Immutable.JS, v5 doesn't have such thing. The dirty workaround makes white- and blacklisting impossible.

Redux-persist is local-only solution

In most apps state should be stored on both, device and the server. In this scenario, we not only need to save some CPU cycles, but also network, which mobile users may have to pay for. Since we need to write API calls to the server anyway, it's a good idea to add local persistence to the very same functions that send and receive data from the server. Syncing over the network should be optimized and well tested. Local persistence gets that for free.

A nice idea for some apps is preserving the state directly in a serverless solution like Firebase or MongoDB Atlas/Stitch. If the state is synchronized there, we don't need to use AsyncStorage for anything else than keeping information about logged in user.

How to make things work without automatic persistence library

AsyncStorage is a very simple key-value store that preserves data between subsequent runs of your app. The API has three main functions:

AsyncStorage.setItem("key", "value")

AsyncStorage.getItem("key")

AsyncStorage.removeItem("key")

Integrating AsyncStorage with Redux via redux-thunk is as easy as writing any request to a backend. Look at documentation of RNSA to see how to do it.

PS. I thought I could add redux-persist to just one subtree of the state so that certain use cases were easier, but it turned out impossible. Not only redux-persist doesn't work with Immutable.JS, but also integrates with Redux so deeply, that all reducers and actions need to be written a little bit differently.

[kalbarczyk]

The new Javascript/React.JS

React.JS is the quintessence of the direction Javascript has taken a few years ago and the result got me into pure amazement. Finally, Javascript is something that can actually compete in every sense with well-established desktop languages. From how the code looks like (a lot less hackish) and its maintainability, to support for more advanced coding paradigms, to one of the best testing and debugging tools. But there is one thing that held me back from learning it - the documentation and learning resources. As a CTO of a startup, I had only a few hours a week to learn new things. It is way too little to switch the whole mindset to the new way of creating apps.

Community dumped in the void all the knowledge I have about web app development. Know javascript? Well... We have ES6 now. CSS? Not quite, we now use SASS/SCSS and CSS3 with things like flexbox and GPU rendered animations. DOM? This is so from the previous decade, React has its own virtual DOM that you program against. HTML? Even that was not spared, say good morning to JSX!

Why not X? Why not Y?

Whoa! Wait a moment, you say, wasn't JS+HTML+CSS combo good enough to make web apps? The simple and brutal answer is: not at all. For ten years now instead of helping people with creating software I was hacking web browsers, learning quirks of poor APIs that were created by consensus-driven W3C. Yes, I've made my living from it, but I think it would be much better if I and thousand others focused on creating user experiences that would match apps by Google on both web and Android! With technologies from above, it is much more possible. Not only we have all web browsers supported, but we can also create native apps for mobile and desktop. All using the same concepts, the same language, the same CSS etc.

OK, but isn't Angular the new better way of solving the problem? Absolutely not! Even with Angular 2.0, it is still the same old way of doing things. The transition from plain jQuery to Angular took me 3 days, then all was the same. Same browser bugs, same JavaScript issues, maintainability problems, project, and code structure, no good practices and ever changing API breaking plugins and making the examples all over the internet irrelevant. Just too much magic injected into the broken run-time environment (web browser). Angular 2.0 tries to fix some of the issues. They switched to Typescript, made big changes in the core. All goes in the good direction, but I don't see the point in waiting when in React everything is already done.

There are some other libraries and frameworks which emerged recently. While all of them are great, a day has 24 hours of which at least 7 should be spent sleeping - we need to choose one. To summarize why I believe React.JS is the better choice, here is a list of things that for me disqualify each of the alternatives:

Vue.JS - Vue took what was not broken from Angular added some React goodies and glued them with concepts from other libraries. I would say it is simpler, less annoying version of Angular. Very easy to switch for an Angular developer. For me, the disadvantage was that Vue is limited to web browsers. No native support for Android or desktop.

NativeScript - a lot of fuss around this language, but I don’t see any other application of it than internal enterprise mobile apps. Plugins that matters are not free, documentation is written in a corporate way. Plus, NativeScript doesn’t support web browsers.

A few challenges

It seems like resources focus on insider people who have watched the development from the very beginning. I have a feeling that documentation starts from somewhere in the middle and ends just before the end. There is a lot of open questions regarding design, best practices, or optimization.

Another barrier is the learning curve. React just doesn't work out of the box. It needs a lot of orchestration tools. To write simple "Hello World!" app we need to know around 10 different tools including an almost completely new language ES6, which is the new Javascript, and JSX. Because no compiler can fully handle the new syntax, we need Babel - a piece of software that converts new syntax to the old one. We need new code editors because the old ones don’t handle ES6 well. We also need Node.JS, NPM, Webpack, Redux or MobX, Immutable.JS and so on. 

Especially annoying is that nowadays javascript is a licensing mess. Nobody knows what licenses mean anymore. You think you are safe because the license is BSD. Later you find out that the library depends on another library which has a language in their license that somebody can sue you for using their code in some circumstances. Now you have a problem. 

React itself. It has PATENTS file which is part of the license. What it means nobody but Facebook knows. They grant you a license to their patents related to React.JS, but if you sue Facebook when they use your patents without your consent, the grant is immediately revoked, so they can easily counter-sue you. I understand where it is coming from (Facebook successfully defended itself from Yahoo! the other day by countersuing). Nevertheless, adding such language to packages that are hard to track is something that for some might be discouraging. I can install a small package that does a little thing and never know that this package depends on React and ten other packages, each with their own license which needs to be scanned by a legal department.

To sum up, entering Javascript world in 2016 is burdened with huge legal uncertainty. 

Fortunately, there are signs that the industry has matured enough to focus on innovation rather than endure in fights over patents. React is used by PayPal, Airbnb, BBC, HP, and others. Meteor switched to React, HP created Grommet on top of it.

Good sources of info

https://teropa.info/blog/2015/09/10/full-stack-redux-tutorial.html

https://camjackson.net/post/9-things-every-reactjs-beginner-should-know

https://github.com/markerikson/redux-ecosystem-links

[kalbarczyk]

My current set-up (with short reviews)

I was pretty happy with my PC from 2008. I had no reason to upgrade anything in 8 years until it started to be pretty unstable in 2016. Ubuntu started to crash often, some graphical glitches occurred from time to time and Apple Cinema Display 23′’ started to disconnect on its own. 

I wanted to make the upgrade as painless as possible. I bought a new monitor totally silent PC and attached old SSD. 

Turned out it wasn’t the best idea. The new monitor with slightly better spec than CinemaDisplay was terrible and PC, while silent and powerful, was poorly supported by Ubuntu. I wasted many hours on tuning the system, with no luck. The biggest problem was the screen. Apple’s 1080p is not the same as Samsung’s 1080p.   

I started to search for something on the higher end and decided to go with 4K screen. I checked my graphics card, despite the fact it wasn’t cheap or old, it didn’t support 4K. I had to take the upgrade to the next level...

The current setup

MacBook Pro 13'' 2017

Despite fact that this was the cheapest MBP on the shelf it's pretty powerful. It can easily handle running simutalnously: - Android Studio - XCode - Simulator x 4 (4 different devices) - one Android Emulator - Visual Code Studio - Chrome - Spotify - Console x 6 tabs - react-native server - few utilities

The biggest pain is Chrome. It has a lot of memory leaks on OSX that were non existent on Linux.

OS X is extremely stable and powerful. I don't see much of a difference between customizability of Linux and OSX. There is bash, console, commands are almost the same, brew is as powerful as apt. There are hundreds of UIs for services and tools unavailable (or poorly supported) under Linux. The OS X GUI is usability mastership.

Overally switching from Linux to OS X is a huge improvment to productivity.

Cons: - Sound card is terrible. There is significant difference between Apple sound card and SoundBlaster Audigy ZS Platinum from 1999 I believe.

LG 27UD88-W

4K 27'' monitor from LG. It's amazing. Period.

Microsoft Sculpt Ergonomic Desktop

It is must have when your hands are bigger than average. In other cases it is still a great solution for pains in your joints.

[kalbarczyk]

Tools to help with Python code quality

Recently we've decided to separate ObjectPath from the Asyncode Framework (ACF) and make it usable for a wider audience as a standalone tool. The problem we faced was that ACF's test suite has been written in BLSL (our own language that ACF interprets), which isn't suitable for Python environment (for which ObjectPath is intended). Therefore, we needed to start from scratch with testing the code base.

The positive thing about starting again was that it's given us a chance to try a few cloud-based tools that help to maintain code in good shape, including TravisCI, Landscape.io and Coveralls.

This post is intended to answer questions about why anyone should do automated code testing including coverage tests and code quality assurance. In other words it's stating the obvious for experienced devs.

The experience all of us has had (the problem)

Why bother with testing at all? The answer to this question is obvious for many. If however you are wondering if it's worth your while, consider the following situation. 

Imagine you have an idea for a cool piece of software. You write something overnight or during the weekend and you're so proud of yourself that the software solves a certain problem and saves hours of time. You're happy the job is done, show results to friends and even start to get requests for new features.

How cool! Isn't it? You have proven your creation is useful not only to you! You add new features, you fix bugs. After some time you run the new version of the software to deal with the problem that we began with in the first place and... it fails!

Why? The thing here is that every alteration to the code results in the software doing something slightly different than before. Consider the following scenario:

Let's say you want to create an intelligent way to add numbers that works also for numbers stored in strings (I have a good explanation why anyone would want to do that, if you ask:) ) and you implement it like this:

def add(x, y): return int(x)+int(y) add(2, 2) -> 4

It works, but after a while you think that it would be cool to support edge cases when add() takes True, False and None. You write something like:

def add(x, y) if x in [True, False, None]: return int(y) if y in [True, False, None]: return int(x) return int(x)+int(y)

And so you have a nice function that deals with all edge cases. Then someone else requests support for floats. Why not?

def add(x, y) if x in [True, False, None]: return float(y) if y in [True, False, None]: return float(x) return float(x)+float(y) add(3.1, 3.2) -> 6.3

You might think at first that you'll get away with using float() here because indeed:

add(2, 2) == 4 -> True

But just a few minutes after the release somebody files a bug because a function stopped working. After a brief investigation it turns out that the person used your function in a different way than you had expected:

add(2, 2) is 4 -> False

Now it's even hard to think about fixing the whole code. It has to be rewritten from scratch.

The time for writing tests

The code above doesn't make sense at all, because we just accepted all requests one after another without any consideration. The problem gets worse and worse, rendering the code complex and extremely hard to fix after some time. That's why before even opening the code editor we should define the scope of the problem, then write how the code should be used (in a real programming language) and only then implement it.

This is called test-driven development. Of course no one can think about every possible use case at the beginning of software development journey, but starting with any test can result in much better code quality. Then before accepting any enhancement request first ask about specific usage of your function. Instead "I need support for False, True and None" requester should provide exact code that requires that. For example:

d={} if random()>0.5: d["a"]=33 add(2, d.get("a"))

d.get() will be either 33 or None in this case.

If you have this piece of code, put add(2, None) is 2 to test suite like Python's unittest package and before any push test if this particular code results aren't broken.

Automating tests - TravisCI

Doing tests before every push can be troublesome. It's normal that sometimes we forget about this, in other cases we think that these aren't harmful changes and we don't do tests. TravisCI can help us with this (and also with a lot more things!).

All we need is to sign in with our GitHub credentials, turn on the Contiguous Integration on repo's that we want to cover, create .travis.yml file in repo and push the code to github. Travis will automatically test what we pushed and generate report. The cool thing here is that we can test our software in many environments at the same time. ObjectPath is tested in Python 2.7, 3.2, 3.3 and PyPy on each push! I don't even need to have PyPy on my box.

If we want to create package and upload it to PyPi it can be done automatically by Travis as well. In comparison to what it is done manually it's game changing functionality.

The code quality - Landscape.io

There are some tools for checking code quality available, but with landscape.io I don't even need to know them - this service has all the best of them. Using the service is as simple as with TravisCI - just sign in with GitHub.

The big drawback of underlying tools is that they are too picky. Comment block of code and your score is some points worse. It also doesn't know some obvious Python tricks, but in the end the quality of ObjectPath's code went significantly up because of automatic checks and easy to read reports.

You can think why to use Landscape instead of relying on PyCharm's integrated code checks. The answer is: do both. PyCharm hint you what to do, but Landscape gives you the overall code quality score and informs you when the score went up or down. Don't underestimate this feature!

Coverage - Coveralls.io

Not that easy to configure in case of Python, but definitely worth it!

Code coverage is very important concept in testing your code. All above tools aren't relevant until you have good code coverage. Because it's little hard to find simple definition of this concept I'll try to craft my own:

Code coverage is a measure that checks how many and which lines of code is tested in unit tests.

Tests are run in a environment that can check what lines of your code are used to run each test and then create rapport - in case of Coveralls nice HTML based report. Red indicates that the line was not used during tests, green is good. Red indicates that you are probably facing the problem from beginning of this article or that it's deprecated piece of code (I found many lines that were not needed anymore in ObjectPath using this tool).

WARNING! Green doesn't indicate that your code is well tested. It just tells that the line was used at least once during tests. Providing relevant tests is still in your hands.

What next?

These tools are just beginning in quality testing that you need to perform. Programming is a discipline that only human can practice and nothing is better for your code than audit by other person. Show your code to friends, ask them if they understand it. When you write commercial code professional code reviews or audits are good option. Always eyes of person who is not involved in the project can find bugs that you would be never aware of!

[kalbarczyk]

ESP8266 GPIO <> pinouts

Mongoose OS doesn't have pinout to GPIO mapping. D0, D1 and so on from the board doesn't mean anything. Here is the mapping from Arduino:

C/C++:

static const uint8_t D0 = 16; static const uint8_t D1 = 5; static const uint8_t D2 = 4; static const uint8_t D3 = 0; static const uint8_t D4 = 2; static const uint8_t D5 = 14; static const uint8_t D6 = 12; static const uint8_t D7 = 13; static const uint8_t D8 = 15; static const uint8_t D9 = 3; static const uint8_t D10 = 1;

JS

let ESP8266_PINS = { "D0": 16, "D1": 5, "D2": 4, "D3": 0, "D4": 2, "D5": 14, "D6": 12, "D7": 13, "D8": 15, "D9": 3, "D10": 1 }

Usage

let pin = ESP8266_PINS.D1; GPIO.set_mode(pin, GPIO.MODE_OUTPUT); GPIO.write(pin, 1);

[kalbarczyk]

Starting a new React Native project

Because Expo has given me much pain and, in a few crucial moments, it failed miserably with sharing my work with a client, I rolled back all projects I'm involved into a pure React Native.

The most annoying issue with Expo is when you upload a project to their server and the client going to the exp:// URI sees a blank app. The only way to get the app to work again is to change the name of the project and upload it again. Other major issues are: * rendering on Android is broken in certain cases * fonts sometimes don't work * exporting an app to an IPA file (required by automated testing services like BrowserStack) is impossible * Expo doesn't work with native extensions. Some of them deliver much better experience than those from Expo library (see Firebase - Expo uses JS SDK) and many provide functionality which Expo simply doesn't offer.

Nothing of the above is an issue when pure RN is used so when I start a new project I eject project immediately after creating it.

Steps

$ npm install -g create-react-native-app $ create-react-native-app MyProject $ cd MyProject/ $ yarn run eject ? How would you like to eject from create-react-native-app? React Native: I'd like a regular React Native project. We have a couple of questions to ask you about how you'd like to name your app: ? What should your app appear as on a user's home screen? MyProject ? What should your Android Studio and Xcode projects be called? MyProject $ yarn start

Then I go to XCode, open MyProject/ios/MyProject.xcodeproj and click run. Not very complicated, but it makes life much easier in later stages of the project.

[kalbarczyk]

Indentation reviewed

When you browse my Github repos it is apparent that I am a big fan of using tabs over spaces. The reason for making up my mind years ago in favor of tabs was the ability for teammates to work on the code using their preferred number of spaces per indent. In languages such as C, Python, PHP, or Java, code with 2, 4, or even 8 spaces per indent looks perfectly fine. There are few places where lines must be split or tuned by hand to make code readable.

However, things have changed. Nowadays we use frameworks, languages, and tools which mix many syntaxes or even coding paradigms. The best example here is React.JS in which a markup language is embedded into ES6, which is a mix of class and functional style programming language.

To make React code readable, some sacrifices are needed and for me hardcoding indents with spaces is the biggest one.

My current way of indenting a React component (React Native in this case):

import React, { Component } from 'react' import { StyleSheet, Image, Text, View, TouchableHighlight, Platform } from 'react-native' const Ball = props => { const ball = props.def.toJS() const bkg = {backgroundColor: ball.color} const striped = ball.suit === "striped" return ( <TouchableHighlight onPress={() => props.onPressFn(ball.value)} disabled={ball.cleared} style={[ styles.Ball, (striped ? {} : bkg), (ball.cleared ? styles.BallCleared : {}), (ball.current ? styles.current : {}) ]}> <View> <View style={(Platform.OS === "android" && striped ? styles.androidFix:{})}/> <View style={[ styles.BallNo, (striped ? [styles.striped, bkg] : {}) ]}> <View style={(striped ? styles.BallNo : {})}> <Text style={styles.BallNoText} allowFontScaling={false}> {ball.value} </Text> </View> </View> </View> </TouchableHighlight> ) } const ballSize = 60 const styles = StyleSheet.create({ Ball: { alignItems: 'center', backgroundColor: 'white', justifyContent: 'center', borderRadius: ballSize, margin: 2, height: ballSize, width: ballSize, overflow: "hidden" }, ... }) export default Ball

The code above looks pretty when spaces per tab are set to 2. What happens when you have this option set to 8 can be seen on Github.

Rules

I use 2 spaces per indent

I write one dependency per line in import statements

I separate logical parts of the code with blank line (mimics paragraphs)

I write XML tag attributes in separate lines and indent them so that they're aligned

If an XML tag attribute value needs indentation (e.g. I pass an inline function or JSON object), I start indentation from the column where = is. When the resulting indentation is too wide, I extract value to a variable.

PS. Doing indentation in Tumblr editor is a nightmare. If you want to post pretty JSX code here, make sure that all indents are spaces, all

[kalbarczyk]

Installing NVM + PM2

curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.31.0/install.sh | bash (go to https://github.com/creationix/nvm for newest version)

nvm install 5.0

nvm use 5.0

npm install pm2 -g

[kalbarczyk]

nodejs http listen port 80 without sudo using iptables

With iptables you can e.g. redirect TCP requests on port 80 to a different port. The reason this comes in handy is that listening on port numbers below 1000 requires elevated permissions on Linux.

sudo iptables -t nat -I PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080

sudo to run iptables with elevated permissions.

-t table to insert new rule

-I insert this rule before all other rules in the chain

-p protocol

--dport source port

-j jump to target

--to-port destination port

If this problem made you run your scripts with sudo, now you can stop that.

Source: http://www.cyberciti.biz/faq/linux-port-redirection-with-iptables/

[kalbarczyk]

Setting up DigitalOcean Ubuntu user like it was on AWS

Create droplet

Pick Ubuntu 14.04 or 16.04 x64 image

Add your SSH keys to the droplet

Pick short and nice name for the droplet (it will show up in the bash prompt like root@ubuntu-2gb-nyc2-01:~$ which is not most convenient)

ssh root@IP_ADDRESS

In console write: $ adduser --disabled-password --gecos "" ubuntu

cp -r /root/.ssh /home/ubuntu/.ssh

echo "ubuntu ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers

chown -R ubuntu:ubuntu /home/ubuntu

Edit /etc/ssh/sshd_config:

PermitRootLogin no

rm -R /root/.ssh

[kalbarczyk]

Upload Folders into Amazon S3 from Shell and Make Them Public

Paste this into your terminal:

sudo apt-get install s3cmd s3cmd --configure touch deploy.sh echo '#!/bin/sh s3cmd sync -r folder1 folder2 s3://acf.acimg.eu/ s3cmd setacl s3://acf.acimg.eu/folder1/ --acl-public --recursive s3cmd setacl s3://acf.acimg.eu/folder2/ --acl-public --recursive' > deploy.sh

Replace folder1 and folder2 with your folders, acf.acimg.eu with your bucket name and do:

sh deploy.sh

[kalbarczyk]

Set up your SSH to connect automatically with servers

The problem

If you have AWS EC2 Linux instances (or any other server that uses RSA key authentication), you probably connect to it with:

ssh -i key.pem ubuntu@example.com

This way used to work for me as long as I had one or two instances in the same region and the same key was used to connect to both of them. Then I created an instance in APAC zone, made some changes to DNS and started having problems remembering what domain + key combination is used to connect to a particular instance.

I searched the Internet to find a nice alternative solution and, having tried a bunch of them, SSH configuration proved to be the easiest and most scalable way to solve the problem.

The solution

Create a file in ~/.ssh folder named config (if not already there):

touch ~/.ssh/config

Say you have an Ubuntu instance in EU Ireland. Add the following:

Host eu     HostName eu.example.com     User ubuntu     IdentityFile ~/.ssh/keys/eukey.pem     ServerAliveInterval 30

at the end of the file (watch for indentation!). That will make:

ssh eu

equivalent to:

ssh -i eukey.pem ubuntu@example.com

Additionally, setting ServerAliveInterval to 30 prevents ssh sessions with EC2 instance from hanging.

Debian example

Host debian     HostName sgp.example.com     User root     IdentityFile ~/.ssh/awskeys/sgpkey.pem     ServerAliveInterval 30

ssh debian

At Asyncode we currently adhere to the following server naming convention:

<SERVICE_NAME>-<AWS_REGION>-<NO>

for example ssh MONGODB-AP-1.

At the moment we have 5 EC2 instances and the solution already saved us hours.

Know better hacks? Comment down below!