harpreetsb

[harpreetsb]

New Blog

Visit Here for by new blog posts

[harpreetsb]

Track Me Not: DNT & responsibility for developers

The Do Not Track feature was introduced long ago in the browsers. The browser would send a header flag and the website/application should do the rest.

But in the market for tracking users and providing personal browsing experience, the website/portal owners never followed the rule to not track user when browser asked for it.

This blog on Medium.com guides you to enable/disable the DNT option in different browsers.

Since I use firefox as my primary browser, below is the devtools screenshot for a GET request sent to google.com.

You can see the DNT header vaue sent from browser.

Below is another devtools screenshot for nytimes.com

Even though the DNT header flag was sent, the website still loaded all the trackers/adverisements. Below is screenshot of all trackers blocked with ghostery.

It is not just about the nytimes.com. Most of the webistes to not follow the DNT request.

Responsibility as a developer/IT Manager.

Someone who is a developer or an IT Manager should follow this request as a global standard. Once you start doing it, others will follow.

What to block?

Well once you have DNT header flag you can block the following:

All third party scripts that track user.

Any cookies created by third party trackers.

This is all for now. In case I have missed anything, Please do let me know more in comments.

Thank you.

[harpreetsb]

Elasticsearch: Searching on nested data

I have been working with Elasticsearch(mentioned as ES) for almost 2 years now. Have used it on 3 projects, but I would say i am still learning it. There have been times when I reviewed my old ES queries and found them incorrect. Here is portion of my experience i would like to share.It is easy to search ES with root level data but what abut nested data.

Elasticsearch provides nested relationship, here is how we can use it.

Nested data is the data that us related to the main document and can be searched through. Example: product has categories. the chances are a single product can belong to different categories. Each category can have its own details like slug, image, description.

Similarly main document in ES can have list of hospitals and each hospital offers various treatments. So there are chances where you need to search products by category and hospitals by treatment.

For such kind of relation within document we need to specify mapping to ES for a particular document type. Example mapping for hospital

{ "properties":{ "hospital_address1":{ "type":"string" }, "hospital_address2":{ "type":"string" }, "hospital_description":{ "type":"string" }, "hospital_id":{ "type":"integer" }, "hospital_name":{ "type":"string" }, "hospital_slug":{ "type":"string", "index":"not_analyzed" }, "hospital_status":{ "type":"string", "index":"not_analyzed" }, "treatments":{ "type":"nested", -- WE SAY THAT TREATMENTS ARE NESTED IN HOSPITALS "properties":{ "child_of":{ "type":"integer" }, "hospital_id":{ "type":"integer" }, "id":{ "type":"integer" }, "keywords":{ "type":"string" }, "status":{ "type":"string", "index":"not_analyzed" }, "treatment_id":{ "type":"integer" }, "treatment_name":{ "type":"string" }, "treatment_price":{ "type":"long" }, "treatment_slug":{ "type":"string", "index":"not_analyzed" }, "treatment_status":{ "type":"string", "index":"not_analyzed" } } } } }

Now the search request would be as follows

// THIS IS A SAMPLE CODE TO SHOW HOW THE QUERY WOULD LOOK LIKE // this code is of a sample boolean query { "sort": [ { "hospital_name": { "order": "asc" } } ], "query": { "bool": { "must": [ { "nested": { // -- have look here "path": "treatments", // -- and here "query": { "bool": { "must": { "term": { "treatments.treatment_slug": "cardiology" } } } } } }, { "term": { "hospital_status": "active" } } ], "minimum_should_match": 2 } } }

read more here

Thank you.

[harpreetsb]

Elasticsearch 2.0, Accessing via public IP

Hi, so as the part of team, I was responsible for upgrading elasticsearch. In our project we were feeding elasticsearch with processed data from mysql so it was not important to save data. So I simply installed ES2.0 and since we are still developing the project we were accessing ES via public IP. But as soon as we upgraded we could not longer access ES via http://<public_IP>:9200. This is because ES2 binds to localhost by default.

Solution

edit file /etc/elasticsearch/elasticsearch.yml

update the following value for network.host: to

network.host: 0.0.0.0

restart you elasticsearch instance and you are good to go.

NOTE: This setting is only for test servers and not at all recommended for production servers.

[harpreetsb]

Creating "Do You Mean" spelling suggestion with php pspell extension

In my recent work we had requirement for creating search engine like spelling suggester. I did some research and found a quick php extension that does this easily for us. Though it has some problems some times for a few words, but usually it wont occur.

I found a php extention called pspell. Its the spelling suggester for php. Note this requires aspell library to work.

Installation

sudo apt-get install libpspell-dev sudo apt-get install php5-pspell sudo apt-get install aspell-en

Once installed restart apache

sudo service apache2 restart

Sample code and test.

read comments in code for good walk through.

<?php $pspell_link = pspell_new("en"); //load the dictionary if (!pspell_check($pspell_link, "clokc")) { // check for work in dictionary, if not found $suggestions = pspell_suggest($pspell_link, "clokc"); // get suggested words foreach ($suggestions as $suggestion) { echo "Possible spelling: $suggestion<br />"; // show suggestions } } ?>

For using spell suggest in sentences you can use this function below.

function spellSuggest($string) { // Suggests possible words in case of misspelling $config_dic = pspell_config_create('en'); // Ignore words under 3 characters pspell_config_ignore($config_dic, 3); // Configure the dictionary pspell_config_mode($config_dic, PSPELL_FAST); $dictionary = pspell_new_config($config_dic); // To find out if a replacement has been suggested $replacement_suggest = false; $string = explode(' ', trim(str_replace(',', ' ', $string)));// repalce any commas with spaces and explode into an array foreach ($string as $key => $value) {// loop through each word if(!pspell_check($dictionary, $value)) {// check againt existance in dictionary, if not found $suggestion = pspell_suggest($dictionary, $value); //add word for suggesstion // Suggestions are case sensitive. Grab the first one. if(isset($suggestion [0]) && (strtolower($suggestion [0]) != strtolower($value))) { $string [$key] = $suggestion [0]; $replacement_suggest = true; } } } if ($replacement_suggest) { // We have a suggestion, so we return to the data. return implode(' ', $string);// combine the string back } else { return null; } }

Usage

echo spellSuggest("This is amazingly asewome");

output:

This is amazingly awesome

Try playing around with it. leave a comment if you like it or hate it.

[harpreetsb]

Securing Logins

This all started when I came to know that my linkedin password was hacked, being a developer i knew few ways to secure/encrypt my password. Most of the noob developers use md5 or hash. I kept thinking but never got time to google it up.

But one of my developer friend from Egypt shared a link with me on twitter. I read and asked several dumb question to the author, you can see few comments by name "bluepiccaso" that is me.

here is bit shorter and exact working for securing your passwords. Please note that this will not a 100% solution for securing your passwords but its good.

Lets take an example

$actualPassword = 123456 $securePassword = md5($actualPassword); // which is e10adc3949ba59abbe56e057f20f883e

The trick is to hash a password not just time but multiple times. just to make you understand here is a quick example

$pwd = 123456; for($x=1;$x The solution i found was using <a title="crypt" href="http://php.ss23.geek.nz/2011/01/12/Using-crypt.html" data-blogger-escaped-target="_blank">CRYPT()</a>. The php `crypt()` function simply encrypts your password. here is simple example(as from the article linked above) ```php // You would of course, get this from $_POST['Password'] or similar when registering an account, or changing a password. $Password = 'MySuperSecretPassword123'; $HashedPassword = crypt($Password); echo "We've generated a new hashed password of: {$HashedPassword}, from {$Password}."; // that would echo /* We've generated a new hashed password of: $6$tuGPKBZX$eRY4lydz6jUzVPVDZYz3M/JIiEyqqgfDd7MgpkByvtyPuDdZDYE9AVYF1u9ND6zdrJvCOwLEmsIQ4g64/GMQi0, from MySuperSecretPassword123.*/

But now, where is the security?, how do i know how much times has it hashed the password.

the crypt function takes one more optional parameter: a salt string to base its hashing on.

here's how

$Password = 'SuperSecurePassword123'; // These only work for CRYPT_SHA512, but it should give you an idea of how crypt() works. $Salt = uniqid(); // Could use the second parameter to give it more entropy. $Algo = '6'; // This is CRYPT_SHA512 as shown on http://php.net/crypt $Rounds = '5000'; // The more, the more secure it is! // This is the "salt" string we give to crypt(). $CryptSalt = '$' . $Algo . '$rounds=' . $Rounds . '$' . $Salt; $HashedPassword = crypt($Password, $CryptSalt); echo "Generated a hashed password: " .$HashedPassword . "\n"; /* As seen above the $rounds is the value of number of times the password should be hashed. $Algo is 6 i.e. "$6$" for sah512, u can use 1 for md5 */

Authenticating users

So how do you really us the code above.

here is the explaination

when user registers with a password

$Password = ‘SuperSecurePassword123′;

what you can do is

$Salt = uniqid(); $Algo = ’6′; $Rounds = ’5000′; $CryptSalt = ‘$’ . $Algo . ‘$rounds=’ . $Rounds . ‘$’ . $Salt; $HashedPassword = crypt($Password, $CryptSalt);

and save the $Hashed Password to database field.

while doing the login you would simply check its as

if (crypt($Password, $HashedPassword) == $HashedPassword) /* where $Password is the password that user used in the Login password field and $HashedPassword is fetched from database to its corresponding username or email(what ever you use for the login credentials)*/

In the code above

if (crypt($Password, $HashedPassword) == $HashedPassword)

do not get confused, the crypt would return the same hash code for which i was created.

here is another sample code to test this

$Password1 = 'WrongPassword'; $Password2 = 'SuperSecurePassword123'; $HashedPassword = '$6$rounds=5000$4d2c68c2ef979$PZTAkwfvCZN0nT4La/0eNNKLt43w1B7DUkFNc9t1bnOG0OJRESnDa1E1H812TZ3CiBqd2qrcFrz2pk/kqpAy3/'; // the hash created for $password2 // Now, what about checking if a password is the right password? if (crypt($Password1, $HashedPassword) == $HashedPassword) { echo "Hashed Password matched Password1"; } else { echo "Hashed Password didn't match Password1"; } if (crypt($Password2, $HashedPassword) == $HashedPassword) { echo "Hashed Password matched Password2"; } else { echo "Hashed Password didn't match Password2"; }

copy paste this simple code and enjoy.

I hope this helps loads of new developer who seek knowledge.

below are some links to related articles

Php Crypt

##php Clone of Ptacek’s Article on Hashing

PHP Security Consortium Article

sha512 algo

comments and feedbacks are welcomed

thank you