letsresonate

[letsresonate]

JSON Text Compactness  for Database Storage

Today, we'd like to share a simple innovation in how we structure the JSON documents that are stored in the database - we coalesce the redundant / repeated information in the JSON doc to come up with a compact JSON document that is reduced in size without losing any textual readability in the JSON doc.  Our compactness technique is somewhere between the JSON Compression techniques and JSON Normalization techniques (IMHO, JSON Compression focuses on serializing json for storage and loses text readability and JSON Normalization techniques normalize the JSON into flat dataframe structures which is somewhat different from what we are trying to achieve)

Though, the redundancy and repeatedness of the information that we've solved for is unique to our usecase, we do believe similar patterns can be used in different situations to compact JSON documents by identifying redundancy and repeatedness in these usecases as well.

We've used AWS Dynamo DB as our database - it has worked well for the usecases that we've had and made development a breeze.  We've also experimented the same JSON document storage on the Azure Cosmos DB which also has worked like a charm.

So, without further ado, lets setup our example and see how we compact the JSON document.

Usecase

Lets say we have a Songs table in the database that stores the details about the different Songs that the users have played / are available in the library.

Our app is available in multiple marketplaces (US, Canada, UK etc) and our JSON document stores the song details for each of these marketplaces in the same JSON document (simplifies cross region song sharing usecases - a consumer in US shares a song with a friend in the UK).

From experience, we've observed that for a large majority of the songs, the song attributes do not differ between the different marketplaces. In some cases the song attributes do differ in different marketplaces. Lets suppose we have the following Song JSON document with a single albumID attribute that is a string type and we have 7 marketplaces. In the example below, marketplace1-4  have same album Id (AlbumID1) and marketplace5-7 have a different  album Id (AlbumID2).

{    "songID":  <system generated PartitionKey unique across all marketplaces>    "albumID":    {        "<marketplace1>":  "<albumID1>",        "<marketplace2>":  "<albumID1>",        "<marketplace3>":  "<albumID1>",        "<marketplace4>":  "<albumID1>",

       "<marketplace5>":  "<albumID2>",        "<marketplace6>":  "<albumID2>",        "<marketplace7>":  "<albumID2>"    } }

We compact the Song JSON document's albumID attribute by defining a defaultSet that is the largest set of marketplaces with the same value and then save the defaultSet and defaultValue for these marketplaces. The marketplaces that are are different are kept as is. The json doc with the compacted albumID attribute would look like the following:

{    "songID":  <system generated PartitionKey unique across all marketplaces>    "albumID":    {        "defaultSet":        [            "<marketplace1>", "<marketplace2>", "<marketplace3>", "<marketplace4>"        ],        "defaultValue": "<albumID1>",        "<marketplace5>":  "<albumID2>",        "<marketplace6>":  "<albumID2>",        "<marketplace7>":  "<albumID2>"    } }

There is an additional optimization that could be made - defining ranked default sets with the repeat frequency - but we did not feel the need to implement these.

{    "songID":  <system generated PartitionKey unique across all marketplaces>    "albumID":    {        "defaultSet1":        [            "<marketplace1>", "<marketplace2>", "<marketplace3>", "<marketplace4>"        ],        "defaultValue1": "<albumID1>",        "defaultSet2":        [            "<marketplace5>", "<marketplace6>", "<marketplace7>"        ],        "defaultValue2": "<albumID2>"    } }

Performance

Our app is available in 32 marketplaces and each song has approximately 16 attributes that are to be duplicated for each marketplace, the average document sizes for each song JSON document came out to ~ 405KB (song contains a bunch of song metadata specific to each marketplace etc). With the JSON Text Compactness algo, we reduced the JSON doc size to ~64 KB without any noticeable impact in serialization / deserialization performance.

Implementation

In terms of implementation, we defined a java class with the following interface (this is for storing String value type):

class StringStringDefaultMap {    

  // Serialize constructor that serializes a <marketplace, albumId> map to a compact representation   public StringStringDefaultMap(CaseInsensitiveMap<String, String> valueMap, Context context);

  // Deserialize constructor that deserializes a dynamo db attribute value compacted map to a <marketplace, albumId> map   public StringStringDefaultMap(AttributeValue stringAttributeValueMapAttrVal, Context context);

  // Get an attribute value for the compacted map represented by this class - it will create the JSON document per the DynamoDB JSON format where type is encoded into the json   public AttributeValue getMapAttributeValue(Context context);  

  // Get the <marketplaceId, albumId> value map represented by this class   public CaseInsensitiveMap<String, String> getValueMap();

  // Get the default <marketplace, albumId> key value pairs - albumId should be the same for all the marketplaces in this map since its the default map   public CaseInsensitiveMap<String, String> getDefaultMap()

  // Get the default keyset for the compacted map   public Set getDefaultKeySet(); }

The implementation for the compacted attributes for AWS Dynamo DB encodes the type into the JSON document as required by the DynamoDB JSON format. This implementation also reuses the Dynamo DB String Set type to store the defaultSet. here is what the above example would look like:

{  "songID": {    "S": "<system generated PartitionKey unique across all marketplaces> "  }  "albumID":  {    "M": {      "defaultSet1": {        "SS":        [          "<marketplace1>", "<marketplace2>", "<marketplace3>", "<marketplace4>"        ],      }      "defaultValue1": {        "S": "<albumID1>",      }      "<marketplace5>":  {        "S": "<albumID2>",      }      "<marketplace6>":  {        "S": "<albumID2>",      }      "<marketplace7>":  {        "S": "<albumID2>"      }    } }

In this example, the album ID value type String so we used a StringStringDefaultMap. There are also implementations for the different types that DynamoDB supports such as

<String marketplace, Integer attribute value> // Dynamo DB's number attribute type <String marketplace, List<Map<String, Object>>> // Dynamo DB's List Attribute Type where each list element is map attribute type <String marketplace, List<String>> // List<String> attribute type <String marketplace, List<Object>> // List<Object> attribute type <String marketplace, Set<String>> // String Set attribute type <String marketplace, Map<String,String>> // Map<String, String> attribute type

Another trick we had to implement is to replace marketplace specific references in an attribute value with a placeholder for storage in the database and resolve these at deserialization time. For example, the following data urls were coalesced to a placeholder data url:

https://www.letsresonate.net/us/albums?id=<albumId1> https://www.letsresonate.net/ca/albums?id=<albumId1> https://www.letsresonate.net/uk/albums?id=<albumId1> => https://www.letsresonate.net/{PLACE_HOLDER}/albums?id=<albumId1>

Usage

The code is shared in the repository: https://github.com/resonancedeveloper/JSONTextCompactness

The Tests in the TestDefaultMaps demonstrate on how to use each of the default map classes and serialize / deserialize into dynamo db json. Here is one such example:

   @Test    public void testStringStringDefaultMap() {

       // test default set        int defaultVal = 100;        int defaultThreshold = 10;        Set<String> defaultSet = new HashSet<>();        Set<String> countryCode = new HashSet<>();        CaseInsensitiveMap<String, String> countryCodeValueMap = new CaseInsensitiveMap<>();        for (int i = 0 ; i < 30 ; i++) {            countryCode.add("cc"+i);            if (i < defaultThreshold) {                countryCodeValueMap.put("cc" + i, "value" + defaultVal);                defaultSet.add("cc" + i);            } else {                countryCodeValueMap.put("cc" + i, "value" + i);            }        }

       Map<String, AttributeValue> attrValMap = new HashMap<>();        attrValMap.put("defaultSet", new AttributeValue().withSS(defaultSet));        attrValMap.put("default", new AttributeValue().withS("value"+defaultVal));        for (int i = 0 ; i < 30 ; i++) {            if (defaultSet.contains("cc"+i)) {                continue;            }            attrValMap.put("cc"+i, new AttributeValue().withS("value"+i));        }

       // serialize        Assert.assertEquals(new AttributeValue().withM(attrValMap), new StringStringDefaultMap(countryCodeValueMap, context).getMapAttributeValue(context));

       // deserialize        Assert.assertEquals(countryCodeValueMap, new StringStringDefaultMap(new AttributeValue().withM(attrValMap), context).getValueMap());    }

[letsresonate]

This is the end, beautiful friend!

All things, great or otherwise, come to an end. It is with bittersweet emotion that we announce that we are shutting down Resonance (Let's Resonate).  

We had a great run - started from nothing and in 3 years & 3 months we became a decent social music player with some ardent fans. While we have great ideas for Resonance (Let's Resonate), some of which were actively in the works, the reality is that we've had limited success in user acquisition and retention.

What does this shutdown mean for a customer?

We're pulling the App from the App Store

Existing apps on user's devices would work and data would be available for the next ~30 days - we'll be shutting down the backends on November 20th, 2021

Farewell from the folks at Resonance (Let's Resonate)!

We'd like to sign-off in a style that is uniquely Resonance - a Sing Along from "The End" by The Doors:

"This is the end Beautiful friend

This is the end My only friend, the end

Of our elaborate plans, the end Of everything that stands, the end"

Send us your love and farewell wishes at (support@theresonancelabs.com)

And for one last time, Let's Resonate!

[letsresonate]

We’ve updated Resonance with the latest Facebook SDK!

Today, we've updated Resonance with the latest Facebook SDK!

We encourage you to download the latest version of Resonance and let us know if there are any issues! (support@theresonancelabs.com)

Download Resonance

Let's Resonate!

[letsresonate]

An all new Resonance (Let's Resonate) is now available!

We're happy to announce that we've updated Resonance today with a brand new set of functionality which we know you'll love.

We've updated the Now Playing Music Player, enhanced the now playing UI by adding the now playing playlist, artist and album details and added quick actions to create Sing Alongs, Comments, Hearts, Resonances and Shares from the Now Playing Music Player.

Here are the highlights from the new release:

Improved Now Playing Music Player

In an earlier update we discussed how we had decluttered the Music Feed UI for usability and intuition, essentially moving from a design philosophy of “Tap For More” to “Scroll For More” and in another update, we updated the Library UI using the same philosophy.

We've now updated the Now Playing Music Player with a brand new UI using scroll views and is now consistent with the rest of the app.

We've implemented inline expansion of clipped text by Double Tap - a simple fix with a large usability impact! The inline expansion has been implemented in Now Playing and Library UIs and will be implemented in the Friends Music Feed in coming weeks.

Music Details on the Now Playing Screen

While the UI updates are significant, the highlight of the release is the integration of Apple Music in the Now Playing Music Player, similar to the earlier updates we had done for the Library. We search the Now Playing Song's Playlists, Artists and Albums details on Apple Music and update the Now Playing Media Player to enable an improved Music Player experience. Users can now quickly navigate to a song's album / artist and discover the inspiration behind the music!

Artists details on the Now Playing Music Player display the artist - including Resonance Recommended Songs and the artist's complete discography.

Now Playing Music Player Artist Details:

Albums in the Now Playing Music Player now display the album description which gives a unique insight about the album. The albums in Now Playing Music Player also display the songs from the album that are not in the library to make the complete album available to the user.

Now Playing Music Player Album Details:

Playlists details on the Now Playing Music Player display the playlist description and playlist details on the Now Playing Music Player.

Now Playing Music Player Playlist Details:

Quick Actions

We've updated the Now Playing Music Player with the quick actions from the Friends music feed. Heart the songs you love, comment and share songs from the Now Playing Music Player screen.

We believe users will like Resonance's new immersive Now Playing Music Player!

We encourage you to download Resonance and try it out!

Download Resonance

We'd be happy to listen to what's working and what isn't so that we can improve. (support@theresonancelabs.com)  

Let's Resonate!

[letsresonate]

Summer Fun

It's Friday, and a summer weekend is upon us. Soak in the summer fun by listening to a collection of feel-good summer hits in the "Summer Fun" playlist on Resonance! This weekend, we've picked music from the likes of Demi Lovato, Lana Del Rey, Hot Chelle Rae and more. Wishing you a happy summer weekend!

Let's Resonate!

[letsresonate]

Feel Good Summer Weekend!

It's Friday, and the weekend is upon us. Celebrate the summer weekend with music by listening to our "Feel Good Summer Weekends" playlist on Resonance. This weekend, we've picked music from the likes of Clean Bandit, Hozier, Carrie Underwood and more. Wishing you a sunny fun-filled weekend!

Let's Resonate!

[letsresonate]

DB / Server upgrades

Folks, 

We've done a major database / sever upgrade at Resonance and things seem to be okay on our side. If you are seeing issues in Resonance, let us know asap! (support@theresonancelabs.com) Thanks

[letsresonate]

Summer Kickoff

It's Friday, and the summer officially starts this weekend with the summer solstice on Saturday. What better way to kickoff the summer than by listening to some essential summer anthems? This weekend, we've picked music from the likes of Kid Rock, Alice Cooper, and more. Happy Weekend!

Let's Resonate!

[letsresonate]

Feel Good R & B Weekend

It's Friday, what better way to kick start your weekend than listen to some Feel Good R & B tunes on Resonance. This week, we've picked R & B tunes from the likes of Justin Bieber, Skip Marley, H.E.R and more. Happy Weekend!

Let's Resonate!

[letsresonate]

Resonance Now Playing widgets are now available!

Sharing and social music are central to the Resonance music experience - we've highlighted the Resonance sharing capabilities in the earlier blog posts - about how you can share music from the app with friends on Resonance or on different social network such as Facebook / Twitter etc or send them as text messages / email etc. An important aspect of the sharing is the recipient's music experience. In this post, we'll look at the small changes we've made on the Resonance website that improve the recipient's music experience in a big way.

When you share a song or a playlist on social networks such as Facebook / Twitter or as text messages / mail - we create a new permalink for the shared music and send this link to the recipient. When the recipient clicks the link - they are taken to a detail page on the website which displays the songs and any custom messages / sing alongs for the shared music item. The shared music detail page has links to ~30 sec song previews that the recipient can listen to.

For example, here is the Resonance's "Rock n Roll Hall of Fame" playlist detail page.

Today we are integrating the "Resonance Now Playing" widget to these detail pages. Resonance Now Playing is a javascript music player that creates a playable playlist for all the song previews in the shared music item. Now, you can playback the entire playlist with a single click instead of playing each item individually.

Another cool thing about this widget is that it can be embedded on any html page easily with any music playlist that you may want to feature.

To feature a playlist on a website, grab the share token from the share permalink add this simple javascript/html code to your html page and viola!

<code> <!-- Import JQuery and Resonance widget javascript on the page - these need to be added to the head of the html page --> <script type="text/javascript" src="https://theresonancelabs.s3.amazonaws.com/letsresonatewebsite/javascript/jquery-1.7.2.min.js"></script> <script type="text/javascript" src="https://theresonancelabs.s3.amazonaws.com/letsresonatewebsite/javascript/widget.js"></script> <!--Create a div with id letsresonatewidget and css class widget - the script will add the widget as a subtree of this div--> <div id="letsresonatewidget" class="widget"> <!--Add a script element with draw widget function call --> <script> <!-- There are a couple of options that you can configure - the showPlaylist option determines whether the initial widget display  shows the complete playlist (expanded / collapse). The showBorder option determines whether to show a black border on the player. -->     var showPlaylist = false;     var showBorder = true;     drawWidget("{SHARE TOKEN}", showPlaylist, showBorder); </script> </code>

In case you'd like to import the widget as an iFrame (this is how we add it to our blog on tumblr) - simply add the following iframe html to the html page. We've also used the copied css to fix the heights / widths of the CSS - ideally the iframe should be resized automatically from javascript but we're running into cross origin resource sharing issues and have opted for this trick for the time being. 

<code>

<!-- Add CSS to the head of the html page or custom CSS on sites such as tumblr / blogger etc-->

<style>     .letsresonatewidgetframe {         width: 600px;         height: 230px;        border: 0;    }     @media (max-width: 599px) {         .letsresonatewidgetframe {             width: auto;             height: 350px;            border: 0;         }     }

</style> <!--Add the iFrame element on the html page with the source as our widhet html iframe page-->

<iframe src="https://letsresonate.net/widget.html?token={SHARED TOKEN}&showplaylist=false&showborder=true" id="letsresonatewidgetframe" class="letsresonatewidgetframe" />

</code>

Here is how the Resonance Now Playing widget for the "Rock n Roll Hall of Fame" looks on an html page when added using an iFrame.

Consider adding the widget on your websites / blogs to share playable music previews with the visitors of your website.

Let us know if you run into issues - we'd be happy to help! (support@theresonancelabs.com)

Let's Resonate!

[letsresonate]

Rock & Roll Hall of Fame - Class of 2020!

Being inducted in the Rock & Roll Hall of Fame is Rock’s highest honor. The inductees are people who shaped the history of Rock & Roll. This weekend, we look at the music of "Rock n Roll Hall of Fame’s Class of 2020". Learn more at www.rockhall.com Happy Weekend!

Let's Resonate!

[letsresonate]

The Contests - Roundup

Singing competitions such as "American Idol", "The Voice" and "America's Got Talent" have propelled many artists to stardom. This weekend, listen to "The Contests - Roundup" - a collection of notable hits from the alumni of these singing competitions. Happy Weekend!

Let's Resonate!

[letsresonate]

Resonance (Let’s Resonate) App Previews are here

We’ve been doing a bunch of fit and finish improvements after the last release (Immersive Search and Improved Library). 

Specifically, we’ve updated the Resonance (Let’s Resonate) product page on the App Store to be a little more friendlier than before - friendlier to both the humans and the machines that are crawling the internet for search engines. The inspiration for this work was the App Store Optimization posts that we came across. 

The App Store images have been updated with captions to help convey the idea being presented in the image. 

The App Store text has been updated to include the features we’ve updated the app with over the last months.  

And we’ve created a number of screen recordings and put them together in an App Preview promo videos to help people discover the app and functionality on the App Store. You can preview them here and on our website. (These are still in iterative review on the App Store and we hope that the App Previews will be live in a few days) 

Feel free to reshare :) 

Thoughts / comments / suggestions? Let us know! support@theresonancelabs.com

[letsresonate]

Kick start your holiday weekend with Feel Good Jams by Resonance

Kick start your holiday weekend with Feel Good Holiday Weekend Jams by Resonance. This weekend, we've picked music from the likes of Taylor Swift, Bruno Mars, Daft Punk and more.

Wishing you a happy weekend!

Let's Resonate!

[letsresonate]

Bust a move with a Dance Music Roundup!

This weekend, take out your dancing shoes, bust your favourite moves and dance away the weekend with our Dance Music Roundup! Happy Dance Weekend!

Let's Resonate!

[letsresonate]

An all new Resonance is now available!

We're happy to announce that we've updated Resonance today with a brand new set of functionality which we know you'll love.

We've updated the library UI, enhanced the search experience and adding music recommendations for artists and albums in the library.

Here are the highlights from the new release:

Immersive Search

A few of the users mentioned that the search results were very limited (a list of songs) and we agree - we implemented basic search functionality on launch with the idea of enhancing it at a later time.

We've now updated Search to be an immersive experience built for music discovery. Your search text is now searched across playlists, artists, albums and songs in the Apple Music Library to deliver a variety of search results. We also, pull in related music content such as Resonance Recommended Songs for an artist and their complete discography to allow users to explore and discover beyond the search results.

For example, searching for events such as "Mother's Day" displays a rich set of Mother's Day playlists.

Playlist Search Results for "Mother's Day":

Searching for a genre such as "Rock n Roll" shows Rock N' Roll playlists that one could spend hours listening to.

Playlist Search Results for "Rock n Roll":

Searching for an artist such as "The Weeknd" displays great results across the board: playlist results such as essentials playlists, inspirations and influencer playlists, song results featuring their top songs, artist results featuring the complete discography and album results showing their top albums.

Search Results for "The Weeknd":

Details for Artist Search Result for "The Weeknd":

We believe users will love the new immersive search experience!

Improved Library

In an earlier update we discussed how we had decluttered the Music Feed UI for usability and intuition, essentially moving from a design philosophy of “Tap For More” to “Scroll For More”. We've now updated the Music Library with a brand new UI using scroll views and is now consistent with the Friends' Music Feed.

While the Music Library UI update is significant, the highlight of the update is the integration of Apple Music in the library. We search the Library Artists and Albums on Apple Music and update the library to enable an improved library experience.

Artists in the library now display the Resonance Recommended Songs for the artist and the complete discography of the Artist.

Library Artists in previous revision:

Updated Library Artists:

Albums in the library now display the album description which gives a unique insight about the album. The albums in the library also display the songs from the album that are not in the library to make the complete album available to the user.

Library Albums in previous revision:

Updated Library Albums:

Playlists in the library are benefitting from the enhanced search capabilities and you can now add songs to playlists by searching on Apple Music instead of being limited to songs in the library (requires Apple Music Subscription)

Library Playlists in previous revision:

Updated Library Playlists:

Search on Apple Music in Create /Edit Playlists:

And to round up our usability and intuition work, we've implemented Double Tap functionality to quickly play a library item.

We encourage you to download Resonance and try it out!

Download Resonance

We'd be happy to listen to what's working and what isn't so that we can improve. (support@theresonancelabs.com)  

Let's Resonate!

[letsresonate]

Mama, we love you! Happy Mother's Day!

Moms are the best! This Mother's Day weekend, celebrate your love for the most special person in your life by sending this emotion packed Mother's Day playlist! Mama, we love you! Happy Mother's Day!

Let's Resonate!