#Apache PredictionIO

[imadeit-davidjanes]

I guess the advantage of this is its integration with existing Apache Big Data services. But Google, Facebook and Microsoft have all open sourced some pretty good ML packages, and there’s only so much attention to go around.

Apache PredictionIO (incubating) is an open source Machine Learning Server built on top of state-of-the-art open source stack for developers and data scientists create predictive engines for any machine learning task. It lets you:

quickly build and deploy an engine as a web service on production with customizable templates;

respond to dynamic queries in real-time once deployed as a web service;

evaluate and tune multiple engine variants systematically;

unify data from multiple platforms in batch or in real-time for comprehensive predictive analytics;

speed up machine learning modeling with systematic processes and pre-built evaluation measures;

support machine learning and data processing libraries such as Spark MLLib and OpenNLP;

implement your own machine learning models and seamlessly incorporate them into your engine;

simplify data infrastructure management.

Apache PredictionIO (incubating) can be installed as a full machine learning stack, bundled with Apache Spark, MLlib, HBase, Spray and Elasticsearch, which simplifies and accelerates scalable machine learning infrastructure management.

[speedysuitfun-blog]

Machine Learning Framework

[releaseteam]

via Twitter https://twitter.com/releaseteam

[nisbieee]

Machine Learning beyond Python and R !

Author : Chawat

Machine Learning , AI and Big Data are the top technologies today evolving with a rapid pace and almost all of these are implemented in Python or R.

Python is the most popular choice because of it’s simple syntax and large number of Libraries but Python isn’t the best choice because it is dynamically typed which slows it down.

So , what are the other languages available ?

1. Golang

Go was created at Google in 2009. Go is known for it’s Fast-Compilation , Multi Core functionality , Concurrency , C like syntax and Strong typing .

A number of Libraries are available :

These are some of the most popular libraries for general ML and Neural Nets:

     golearn  – https://github.com/sjwhitworth/golearn

     gago        – https://github.com/MaxHalford/gago

     GoNN      ��� https://github.com/fxsjy/gonn

Popular libraries for Natural Language Processing and Decision Forests:

     Word Embedding – https://github.com/ynqa/word-embedding

     CloudForest  – https://github.com/ryanbressler/CloudForest

     RF  – https://github.com/fxsjy/RF.go

Libraries for Image Processing and Data Visualisation:

     bimg            – https://github.com/h2non/bimg

     go-graph  – https://github.com/StepLg/go-graph

2. Scala

Libraries for general ML and Decision Trees:

   Conjecture         – https://github.com/etsy/Conjecture

   SwiftLearner  – https://github.com/valdanylchuk/swiftlearner/

   ganitha                – https://github.com/tresata/ganitha

   brushfire            – https://github.com/stripe/brushfire

Popular libraries for Data Visualisation:

   MLlib in Apache Spark  – http://spark.apache.org/docs/latest/mllib-guide.html

    Hydrosphere Mist  – https://github.com/Hydrospheredata/mist

    PredictionIO              – https://github.com/apache/incubator-predictionio

    Flink                                –  http://flink.apache.org/

    Spark Notebook     –  http://spark-notebook.io/

Natural Language Processing:

    Breeze          – https://github.com/scalanlp/breeze

    FACTORIE  –  https://github.com/factorie/factorie

    Chalk             – https://github.com/scalanlp/chalk

Just two ?

No , the above two are my favourite choice. There are other languages like Julia , Lua , Java & C++ which have some great ML libraries. So , you might check them as well.

My views on Python

Python is a very powerful language with a lot of libraries and it’s simple syntax. It is thus  very Addictive but it is that simple and easy syntax of the language that makes it slow (dynamic typing).

It’s time to move onto newer technologies.

Disclaimer: I don’t mean to hurt anyone , I have just conveyed my views here.

[mauricebigmoflynn]

Apache PredictionIO

What is Apache PredictionIO? Apache PredictionIO is the open-source platform that allows rebel users to build their own machine learning engines to launch at the web.

Is Apache PredictionIO for experts or beginners? Open source always has a tendency to frighten the uninitiated, who expect to see something like the lines from The Matrix when they boot up the system. It shouldn’t be this way,…

[jastewart]

via Buffer

[salesforceconsultancyservices]

WHY IS SALESFORCE LIGHTNING PLATFORM CRITICAL TO ITS SUCCESS

As Internet of Things has grown its roots in our day to day life, more and more of our data from our devices like phones, cars, home appliances etc. are exposed to the world. According to a recent study, it is found that more than six billion machines have come online forming a connected network (Internet of) Things that generates a daily data of more than 2 quintillion bytes. In simpler terms, this huge amount of data is enough to fill up approximately 58 billion of 32GB iPads daily. It is way beyond the human capacity to process this gigantic data and transform it to create a business impact.

Salesforce IoT cloud provided the companies with a place to store huge data collected from the connected devices. The application of Artificial intelligence to it provides it with a new dimension and many more capabilities. Leveraging Salesforce Einstein and IoT, Sales Cloud can collect data from various connected devices, suggest rules and predict next actions.

With the use of Sales Cloud Einstein, businesses are equipped with the capability to consume this gigantic amount of data and create meaningful customer insights in real-time. By combining Salesforce Einstein along with the IoT data, the businesses will be capable of experiencing a new rise in IoT innovation. Companies can leverage the advantages of Sales Cloud Einstein in CRM from best Salesforce Consulting Companies.

IoT Cloud with PredictionIO

The Salesforce IoT cloud and PredictionIO, both play on the Apache open-source Frameworks like Spark, Storm or Kafka. Salesforce Apps developers will be able to build customized intelligent applications connecting logic with IoT data, all thanks to the Salesforce innovation that brought both these services together.

Analytical Score for IoT Event Data

By bringing the services like IoT cloud and PredictionIO closer, the customers can transfer data at a high speed to AI algorithms, which is trained to arrange for scoring/recording data and  indicating the behaviour of the connected device that needs a service update.

Next Suggested Action

As the IoT Cloud is directly integrated with the primary CRM products, it is connected in such a way that it not only collects IoT data but also detects the conditions where the data from connected devices requires action at employees or customer’s end. By making use of Einstein, the IoT cloud is able to mention the next vital action depending on the situation like suggesting the launch of a particular type of marketing journey based on the observed patterns it has come across, or intelligently bringing an event to a particular representative’s notice who has successfully resolved such cases in the past, & delivered high customer satisfaction.

Optimize the Journey of IoT Device Automatically

Salesforce is working towards using Einstein’s technology to enable customers to optimize their device journey. In future, the customers will be competent to automatically update the rules governing connected device interactions by providing the device journey data into IoT Cloud Einstein. This will be beneficial for the customers as they will be able to restate IoT use cases in order to quickly get to business value.

 HOW WILL INTEGRATION OF SALESFORCE EINSTEIN AND THE IoT TRANSFORM BUSINESS-

In the near future, the SaaS companies are going to completely transform the way we do business. This can be explained by understanding a simple example. Suppose your iPhone suddenly gets the hang. The factory reset is unresponsive and the screen freezes. Your daily job will be affected by this glitch and you will have no option but to wait for a week for the phone to be replaced. The replaced phone might not even include your existing contacts and other phone's data. This will all create a frustrating scenario.

Now imagine that your iPhone has a pre-installed sensor to auto-detect any error. This sensor detects your phone’s glitch and sends your case file in the Apple’s CRM platform. This sensor even directs your case automatically to a customer service agent and provides all your customer information and purchase history to them. Imagine that the agent is equipped with a contact centre solution integrated with its Salesforce Service Cloud with Salesforce’s AI, Einstein.

In this case:

1. The customer service gets notified automatically through your phone without any action taken by you.

2. The Customer Service agent receives your customer information and purchase history that enables him to trigger a replacement order with express shipping.

3. The agent notifies you on a call about the replacement being shipped. The Salesforce Service cloud has speech analytics that can record your call, capture your sentiments and stores information in Salesforce Cloud.

4. Speech Analytics program is able to detect any emotional changes in your voice while you express any concern during the call.

5. Salesforce Einstein can study your current sentiment and compare it with past purchase information and other customer service cases to suggest a relevant action. (eg: if the customer was concerned about the shipment time, the agent can prioritize his shipment for immediate delivery)

6. You receive your replaced phone on the same day- just moments after you notice the glitch.

7. The agent ports all your old phone’s data and contacts into your new phone by accessing your data that was stored in the cloud.

In this example, Salesforce AI Einstein and IOT work together to create an environment in which the customer receives the help from an agent before even noticing the problem. This way the company is able to gain complete customer satisfaction and emerge with the reputation of a Customer First Company. The customer wait time decreases from months to just a few hours- no hostile calls from the customers, no phone hold time, no scouring the customer’s information from their files.

This is just the scenario of the service front of the business. Other departments like Sales reps will also be benefited with this integration of AI with IOT. The CRM files can travel between sales and Service.  IoT could study the customer trends and trigger opportunities for upselling. With the help of AI, the leads can be prioritized based on the probability of success. This way the company’s sales will also increase drastically.

This is the future we are entering into and we are almost here.

Don’t you think it’s time to get the most out of your Salesforce solutions with Einstein! Contact us today for your Salesforce Einstein Integration Services.

[ulyss]

This is a most popular repository list for Scala sorted by number of stars STARS FORKS ISSUES NAME DESCRIPTION 17228 15780 549 spark Mirror of Apache Spark 11255 1820 71 predictionio PredictionIO, a machine learning server for developers and ML engineers. Built on Apache Spark, HBase and Spray.

[ericvanderburg]

Apache Software Foundation Is Bringing Open Source ML to the Masses with PredictionIO

http://i.securitythinkingcap.com/PyWlrt

[jamesmcphee30]

Apache Software Foundation Is Bringing Open Source ML to the Masses with PredictionIO

The Apache Software Foundation is opening up the field of machine learning with its new open source project, PredictionIO. But how are they making it easier for newcomers to learn this devilishly complicated bit of coding? The clever use of templates, of course.

from martinos https://www.linux.com/news/apache-software-foundation-bringing-open-source-ml-masses-predictionio

[technewsandupdatesblog]

PredictionIO, a full stack for building machine learning apps, gets Apache top-level status

PredictionIO, a full stack for building machine learning apps, gets Apache top-level status

The Apache Software Foundation says it’s designating open-source machine learning software first developed by Salesforce.com Inc.

The project is called Apache PredictionIO, and is described as an open-source machine learning server that lets developers and data scientists create predictive engines and services to perform machine learning tasks.

Salesforce quietly acquired PredictionIO back in…

[releaseteam]

via Twitter https://twitter.com/releaseteam

[megatechcrunch]

The right AI solution is the one that fits the skill set of the users and solves the highest-priority problems for the business.

The promises of AI are great, but taking the steps to build and implement AI within an organization is challenging. As companies learn to build intelligent products in real production environments, engineering teams face the complexity of the machine learning development process—from data sourcing and cleaning to feature engineering, modeling, training, deployment, and production infrastructure. Core to addressing these challenges is building an effective AI platform strategy—just as Facebook did with FBLearner Flow and Uber did with Michelangelo. Often, this task is easier said than done. Navigating the process of building a platform bears complexities of its own, particularly since the definition of “platform” is broad and inconclusive. In this post, I'll walk through the key considerations of building an AI platform that is right for your business, and avoiding common pitfalls.

Who will use the platform?

Machine learning platforms are often casually advertised as designed for both software engineers and data scientists. Most of them, however, fail to address both roles well and at the same time. Even worse, they don’t offer enough value to either side to be useful for real work. My experience in building PredictionIO and contributing to Salesforce Einstein AI has helped me understand two distinct groups of practitioners who have diverged sets of requirements in mind.

First, there is the data scientist group. These users usually have a math and statistics background, and are heavy users of tools like R and Python's scientific packages and data visualization tools. This group is responsible for analyzing data and tuning models for the best accuracy, so they’re concerned about whether the platform supports a specific class of algorithms, that it works well with data analysis tools they are already familiar with, and that it integrates with the visualization tools they use. They also want to know what feature engineering techniques it supports, whether they can bring in their own pre-trained models, and so on.

Then there is the software developer group. These users are generally familiar with building web and mobile applications, and are more concerned with whether the platform integrates with the desired data sources, and if the provided programming interfaces and built-in algorithms are sufficient to build certain applications. They want to know how to retrieve model results, whether model versioning is supported, if there is a software design pattern to be followed, and so on.

To implement an AI platform successfully for your organization, you must truly understand your users and do the right heavy lifting accordingly. For example, there are many data scientists who prefer to fine-tune every algorithm parameter manually, but if your users expect out-of-the-box regression that just works, then automated model tuning may become an essential technology of the platform. You want to help these users avoid the hassle of tuning the regularization parameters so they can focus on their top priorities.

Are you solving for simplicity or flexibility?

You may wonder why it is so difficult to build a single AI platform that serves two or more personas well. Can’t we simply offer more functionality on the platform? The problem boils down to the tough choice between simplicity and flexibility. It is more of an art than a science to determine which parts should be abstracted away for simplicity and which parts should be made customizable for flexibility.

For some users, an ideal platform is one that abstracts away all data science details. Many software engineers happily utilize the power of Salesforce Einstein's deep learning APIs to recognize objects in images and to classify sentiment in text without worrying about how the AI model is built, or even which algorithm is being used behind the scenes.

For other users, an ideal platform is one that allows a maximum level of flexibility. Many software engineers like to build completely custom AI engines on Apache PredictionIO. They get their hands dirty modifying the Spark ML pipelines and enjoy the freedom to tailor-make and fine-tune every component— from data preparation and model selection to real-time serving logics— in order to create a unique AI use case.

How do you balance product and engineering decisions?

As an AI platform is adopted by more and more users, many tough but interesting product and engineering decisions are revealed. What should the platform measure? Should it offer built-in metrics? How should it handle the very different requirements for AI R&D, development and production purposes? What’s the cost versus scalability strategy? Should the platform be cloud agnostic? Should visualization tools be part of the platform? To answer these questions most effectively, you must focus on one complete use case for one type of user at a time, starting with the highest priorities for the business.

What Is your multi-layer approach?

Sometimes, the reality is that you do need to construct AI offerings for various types of users. In that case, the separation of offerings must be explicit.

For instance, the Salesforce Einstein artificial intelligence layer has three main components. First, there are several independent services relating to machine learning development. One service is for executing resource-intensive jobs, and is responsible for intelligently allocating and managing distributed computing units to each job. Another service is for scheduling jobs, managing their dependencies, and monitoring the status. These low-level services give data scientists and software engineers the maximum flexibility to build AI solutions in whatever ways they like.

Second, there is an application framework that standardizes the design pattern for some types of common AI applications— specifically in Salesforce’s case, multitenant AI applications. Users will still need to write code, but they write far less of it because many common functionalities are abstracted away. In exchange for some flexibilities, the platform offers resilience and scalability to the AI applications built on top of it.

Third, APIs and user interfaces are provided so that the platform can be useful to users who write very little code, or even no code, to build AI applications.

Conclusion

Companies that do not thoroughly think through their AI strategies often swing from one direction to another one. They are chasing the wind. The growing demand for AI platforms to serve various types of development is inevitable in the foreseeable future, and the right solution is the one that fits the skill set of the users and solves the highest-priority problems for the business.

Continue reading Key considerations for building an AI platform.

from All - O'Reilly Media http://ift.tt/2z66lEj

[repmywind02199]

Key considerations for building an AI platform

Key considerations for building an AI platform

The right AI solution is the one that fits the skill set of the users and solves the highest-priority problems for the business.

The promises of AI are great, but taking the steps to build and implement AI within an organization is challenging. As companies learn to build intelligent products in real production environments, engineering teams face the complexity of the machine learning development process—from data sourcing and cleaning to feature engineering, modeling, training, deployment, and production infrastructure. Core to addressing these challenges is building an effective AI platform strategy—just as Facebook did with FBLearner Flow and Uber did with Michelangelo. Often, this task is easier said than done. Navigating the process of building a platform bears complexities of its own, particularly since the definition of “platform” is broad and inconclusive. In this post, I'll walk through the key considerations of building an AI platform that is right for your business, and avoiding common pitfalls.

Who will use the platform?

Machine learning platforms are often casually advertised as designed for both software engineers and data scientists. Most of them, however, fail to address both roles well and at the same time. Even worse, they don’t offer enough value to either side to be useful for real work. My experience in building PredictionIO and contributing to Salesforce Einstein AI has helped me understand two distinct groups of practitioners who have diverged sets of requirements in mind.

First, there is the data scientist group. These users usually have a math and statistics background, and are heavy users of tools like R and Python's scientific packages and data visualization tools. This group is responsible for analyzing data and tuning models for the best accuracy, so they’re concerned about whether the platform supports a specific class of algorithms, that it works well with data analysis tools they are already familiar with, and that it integrates with the visualization tools they use. They also want to know what feature engineering techniques it supports, whether they can bring in their own pre-trained models, and so on.

Then there is the software developer group. These users are generally familiar with building web and mobile applications, and are more concerned with whether the platform integrates with the desired data sources, and if the provided programming interfaces and built-in algorithms are sufficient to build certain applications. They want to know how to retrieve model results, whether model versioning is supported, if there is a software design pattern to be followed, and so on.

To implement an AI platform successfully for your organization, you must truly understand your users and do the right heavy lifting accordingly. For example, there are many data scientists who prefer to fine-tune every algorithm parameter manually, but if your users expect out-of-the-box regression that just works, then automated model tuning may become an essential technology of the platform. You want to help these users avoid the hassle of tuning the regularization parameters so they can focus on their top priorities.

Are you solving for simplicity or flexibility?

You may wonder why it is so difficult to build a single AI platform that serves two or more personas well. Can’t we simply offer more functionality on the platform? The problem boils down to the tough choice between simplicity and flexibility. It is more of an art than a science to determine which parts should be abstracted away for simplicity and which parts should be made customizable for flexibility.

For some users, an ideal platform is one that abstracts away all data science details. Many software engineers happily utilize the power of Salesforce Einstein's deep learning APIs to recognize objects in images and to classify sentiment in text without worrying about how the AI model is built, or even which algorithm is being used behind the scenes.

For other users, an ideal platform is one that allows a maximum level of flexibility. Many software engineers like to build completely custom AI engines on Apache PredictionIO. They get their hands dirty modifying the Spark ML pipelines and enjoy the freedom to tailor-make and fine-tune every component— from data preparation and model selection to real-time serving logics— in order to create a unique AI use case.

How do you balance product and engineering decisions?

As an AI platform is adopted by more and more users, many tough but interesting product and engineering decisions are revealed. What should the platform measure? Should it offer built-in metrics? How should it handle the very different requirements for AI R&D, development and production purposes? What’s the cost versus scalability strategy? Should the platform be cloud agnostic? Should visualization tools be part of the platform? To answer these questions most effectively, you must focus on one complete use case for one type of user at a time, starting with the highest priorities for the business.

What Is your multi-layer approach?

Sometimes, the reality is that you do need to construct AI offerings for various types of users. In that case, the separation of offerings must be explicit.

For instance, the Salesforce Einstein artificial intelligence layer has three main components. First, there are several independent services relating to machine learning development. One service is for executing resource-intensive jobs, and is responsible for intelligently allocating and managing distributed computing units to each job. Another service is for scheduling jobs, managing their dependencies, and monitoring the status. These low-level services give data scientists and software engineers the maximum flexibility to build AI solutions in whatever ways they like.

Second, there is an application framework that standardizes the design pattern for some types of common AI applications— specifically in Salesforce’s case, multitenant AI applications. Users will still need to write code, but they write far less of it because many common functionalities are abstracted away. In exchange for some flexibilities, the platform offers resilience and scalability to the AI applications built on top of it.

Third, APIs and user interfaces are provided so that the platform can be useful to users who write very little code, or even no code, to build AI applications.

Conclusion

Companies that do not thoroughly think through their AI strategies often swing from one direction to another one. They are chasing the wind. The growing demand for AI platforms to serve various types of development is inevitable in the foreseeable future, and the right solution is the one that fits the skill set of the users and solves the highest-priority problems for the business.

Continue reading Key considerations for building an AI platform.

http://ift.tt/2z66lEj

[doorrepcal33169]

Key considerations for building an AI platform

The right AI solution is the one that fits the skill set of the users and solves the highest-priority problems for the business.

The promises of AI are great, but taking the steps to build and implement AI within an organization is challenging. As companies learn to build intelligent products in real production environments, engineering teams face the complexity of the machine learning development process—from data sourcing and cleaning to feature engineering, modeling, training, deployment, and production infrastructure. Core to addressing these challenges is building an effective AI platform strategy—just as Facebook did with FBLearner Flow and Uber did with Michelangelo. Often, this task is easier said than done. Navigating the process of building a platform bears complexities of its own, particularly since the definition of “platform” is broad and inconclusive. In this post, I'll walk through the key considerations of building an AI platform that is right for your business, and avoiding common pitfalls.

Who will use the platform?

Machine learning platforms are often casually advertised as designed for both software engineers and data scientists. Most of them, however, fail to address both roles well and at the same time. Even worse, they don’t offer enough value to either side to be useful for real work. My experience in building PredictionIO and contributing to Salesforce Einstein AI has helped me understand two distinct groups of practitioners who have diverged sets of requirements in mind.

First, there is the data scientist group. These users usually have a math and statistics background, and are heavy users of tools like R and Python's scientific packages and data visualization tools. This group is responsible for analyzing data and tuning models for the best accuracy, so they’re concerned about whether the platform supports a specific class of algorithms, that it works well with data analysis tools they are already familiar with, and that it integrates with the visualization tools they use. They also want to know what feature engineering techniques it supports, whether they can bring in their own pre-trained models, and so on.

Then there is the software developer group. These users are generally familiar with building web and mobile applications, and are more concerned with whether the platform integrates with the desired data sources, and if the provided programming interfaces and built-in algorithms are sufficient to build certain applications. They want to know how to retrieve model results, whether model versioning is supported, if there is a software design pattern to be followed, and so on.

To implement an AI platform successfully for your organization, you must truly understand your users and do the right heavy lifting accordingly. For example, there are many data scientists who prefer to fine-tune every algorithm parameter manually, but if your users expect out-of-the-box regression that just works, then automated model tuning may become an essential technology of the platform. You want to help these users avoid the hassle of tuning the regularization parameters so they can focus on their top priorities.

Are you solving for simplicity or flexibility?

You may wonder why it is so difficult to build a single AI platform that serves two or more personas well. Can’t we simply offer more functionality on the platform? The problem boils down to the tough choice between simplicity and flexibility. It is more of an art than a science to determine which parts should be abstracted away for simplicity and which parts should be made customizable for flexibility.

For some users, an ideal platform is one that abstracts away all data science details. Many software engineers happily utilize the power of Salesforce Einstein's deep learning APIs to recognize objects in images and to classify sentiment in text without worrying about how the AI model is built, or even which algorithm is being used behind the scenes.

For other users, an ideal platform is one that allows a maximum level of flexibility. Many software engineers like to build completely custom AI engines on Apache PredictionIO. They get their hands dirty modifying the Spark ML pipelines and enjoy the freedom to tailor-make and fine-tune every component— from data preparation and model selection to real-time serving logics— in order to create a unique AI use case.

How do you balance product and engineering decisions?

As an AI platform is adopted by more and more users, many tough but interesting product and engineering decisions are revealed. What should the platform measure? Should it offer built-in metrics? How should it handle the very different requirements for AI R&D, development and production purposes? What’s the cost versus scalability strategy? Should the platform be cloud agnostic? Should visualization tools be part of the platform? To answer these questions most effectively, you must focus on one complete use case for one type of user at a time, starting with the highest priorities for the business.

What Is your multi-layer approach?

Sometimes, the reality is that you do need to construct AI offerings for various types of users. In that case, the separation of offerings must be explicit.

For instance, the Salesforce Einstein artificial intelligence layer has three main components. First, there are several independent services relating to machine learning development. One service is for executing resource-intensive jobs, and is responsible for intelligently allocating and managing distributed computing units to each job. Another service is for scheduling jobs, managing their dependencies, and monitoring the status. These low-level services give data scientists and software engineers the maximum flexibility to build AI solutions in whatever ways they like.

Second, there is an application framework that standardizes the design pattern for some types of common AI applications— specifically in Salesforce’s case, multitenant AI applications. Users will still need to write code, but they write far less of it because many common functionalities are abstracted away. In exchange for some flexibilities, the platform offers resilience and scalability to the AI applications built on top of it.

Third, APIs and user interfaces are provided so that the platform can be useful to users who write very little code, or even no code, to build AI applications.

Conclusion

Companies that do not thoroughly think through their AI strategies often swing from one direction to another one. They are chasing the wind. The growing demand for AI platforms to serve various types of development is inevitable in the foreseeable future, and the right solution is the one that fits the skill set of the users and solves the highest-priority problems for the business.

Continue reading Key considerations for building an AI platform.

from FEED 10 TECHNOLOGY http://ift.tt/2z66lEj

[csemntwinl3x0a1]

Key considerations for building an AI platform

Key considerations for building an AI platform

The right AI solution is the one that fits the skill set of the users and solves the highest-priority problems for the business.

The promises of AI are great, but taking the steps to build and implement AI within an organization is challenging. As companies learn to build intelligent products in real production environments, engineering teams face the complexity of the machine learning development process—from data sourcing and cleaning to feature engineering, modeling, training, deployment, and production infrastructure. Core to addressing these challenges is building an effective AI platform strategy—just as Facebook did with FBLearner Flow and Uber did with Michelangelo. Often, this task is easier said than done. Navigating the process of building a platform bears complexities of its own, particularly since the definition of “platform” is broad and inconclusive. In this post, I'll walk through the key considerations of building an AI platform that is right for your business, and avoiding common pitfalls.

Who will use the platform?

Machine learning platforms are often casually advertised as designed for both software engineers and data scientists. Most of them, however, fail to address both roles well and at the same time. Even worse, they don’t offer enough value to either side to be useful for real work. My experience in building PredictionIO and contributing to Salesforce Einstein AI has helped me understand two distinct groups of practitioners who have diverged sets of requirements in mind.

First, there is the data scientist group. These users usually have a math and statistics background, and are heavy users of tools like R and Python's scientific packages and data visualization tools. This group is responsible for analyzing data and tuning models for the best accuracy, so they’re concerned about whether the platform supports a specific class of algorithms, that it works well with data analysis tools they are already familiar with, and that it integrates with the visualization tools they use. They also want to know what feature engineering techniques it supports, whether they can bring in their own pre-trained models, and so on.

Then there is the software developer group. These users are generally familiar with building web and mobile applications, and are more concerned with whether the platform integrates with the desired data sources, and if the provided programming interfaces and built-in algorithms are sufficient to build certain applications. They want to know how to retrieve model results, whether model versioning is supported, if there is a software design pattern to be followed, and so on.

To implement an AI platform successfully for your organization, you must truly understand your users and do the right heavy lifting accordingly. For example, there are many data scientists who prefer to fine-tune every algorithm parameter manually, but if your users expect out-of-the-box regression that just works, then automated model tuning may become an essential technology of the platform. You want to help these users avoid the hassle of tuning the regularization parameters so they can focus on their top priorities.

Are you solving for simplicity or flexibility?

You may wonder why it is so difficult to build a single AI platform that serves two or more personas well. Can’t we simply offer more functionality on the platform? The problem boils down to the tough choice between simplicity and flexibility. It is more of an art than a science to determine which parts should be abstracted away for simplicity and which parts should be made customizable for flexibility.

For some users, an ideal platform is one that abstracts away all data science details. Many software engineers happily utilize the power of Salesforce Einstein's deep learning APIs to recognize objects in images and to classify sentiment in text without worrying about how the AI model is built, or even which algorithm is being used behind the scenes.

For other users, an ideal platform is one that allows a maximum level of flexibility. Many software engineers like to build completely custom AI engines on Apache PredictionIO. They get their hands dirty modifying the Spark ML pipelines and enjoy the freedom to tailor-make and fine-tune every component— from data preparation and model selection to real-time serving logics— in order to create a unique AI use case.

How do you balance product and engineering decisions?

As an AI platform is adopted by more and more users, many tough but interesting product and engineering decisions are revealed. What should the platform measure? Should it offer built-in metrics? How should it handle the very different requirements for AI R&D, development and production purposes? What’s the cost versus scalability strategy? Should the platform be cloud agnostic? Should visualization tools be part of the platform? To answer these questions most effectively, you must focus on one complete use case for one type of user at a time, starting with the highest priorities for the business.

What Is your multi-layer approach?

Sometimes, the reality is that you do need to construct AI offerings for various types of users. In that case, the separation of offerings must be explicit.

For instance, the Salesforce Einstein artificial intelligence layer has three main components. First, there are several independent services relating to machine learning development. One service is for executing resource-intensive jobs, and is responsible for intelligently allocating and managing distributed computing units to each job. Another service is for scheduling jobs, managing their dependencies, and monitoring the status. These low-level services give data scientists and software engineers the maximum flexibility to build AI solutions in whatever ways they like.

Second, there is an application framework that standardizes the design pattern for some types of common AI applications— specifically in Salesforce’s case, multitenant AI applications. Users will still need to write code, but they write far less of it because many common functionalities are abstracted away. In exchange for some flexibilities, the platform offers resilience and scalability to the AI applications built on top of it.

Third, APIs and user interfaces are provided so that the platform can be useful to users who write very little code, or even no code, to build AI applications.

Conclusion

Companies that do not thoroughly think through their AI strategies often swing from one direction to another one. They are chasing the wind. The growing demand for AI platforms to serve various types of development is inevitable in the foreseeable future, and the right solution is the one that fits the skill set of the users and solves the highest-priority problems for the business.

Continue reading Key considerations for building an AI platform.

http://ift.tt/2z66lEj