enigmaticlearner

[enigmaticlearner]

Real-time Vs Batch analytics

The need of visual analytics has become an important need for the whole world. To aid the daily business aids and to get closer to customers this field has become a good opportunity.

Though it seems like these visualizations are used for mere decorations, in data science aspect these visualizations can be used to gain crucial business insights. Hence three types of main data analytic types can be found.

1.    Predictive analytics – This type is used to predict future patterns by studying data. Techniques like data mining, machine learning, modelling are used to obtain predicted insights.

2.    Interactive analytics – For this type a stored dataset can be queried in an ad-hoc manner to find useful perceptions.

3.    Real-time analytics – Real-time analytics is associated with processing data without loading at a high speed. This is used to retrieve analytical information quickly for crucial applications like credit card transaction monitoring, heart beat rate tracking devices and so on.

With this introduction about analytics types, this blog article focuses on the solution provided to Uber and Lyft Dataset Boston, USA. The original dataset could be found in kaggle website. Batch and real-time analytics were used to assess market situation and also to plan the business for the Uber and Lyft taxi trips.

The research question revolves around quantitative data as the dataset contains mostly about numerical values. Other than that important field names such as product_id, cab_type, etc. are available as shown in below csv file. The original dataset contained 57 values but for the analysis only 13 columns were used as shown in the below diagram.  When analyzing the dataset it was found that the ‘price’ column in the dataset contained ‘NA’ values for Taxi type with a row count of 55000. Therefore the data was preprocessed accordingly and NA value containing rows were removed from the dataset.

 The analytical results were retrieved considering batch analytics and real-time analytics.

Batch analytics

As for this part, databricks platform was used to process the dataset and obtain visualizations as needed. In batch analytics the whole dataset is processed first and then only queries are run unlike in real-time analytics. 

Following are the observations that were studied by me.

 1. Number of rides over time by grouping the product_id

As seen in the diagram, spikes in the graph are not much visible as the number of rides are almost equal for each product_id.

 2. taxi fare for each cab_type and name

According to the visualization, the taxi fares for each cab_type and name have overall equal average prices. Lux Black XL from Lyft cab type and Black SUV from Uber has the height taxi fare.

 3. Interesting observations identified

 1.As a total, Lyft cab type has gained an income more than Uber cabs with a slight increase.

2. The travel counts made by customers to each source has almost equal amounts implying the source rides are equal.

 4.    This graph shows monthly income for each cab type. Lyft cabs has the highest income for both months. In overall both Uber and Lyft cab types has the highest income for month 12.

 Hence the final dashboard for batch analytics is as shown below.

Real-time analytics

The real-time analytics were processed using Siddhi editor provided by Wso2 Company. The csv file was read from siddhi editor and was pushed to the Siddhi Engine. Once the data is set to be processed the data consuming can be seen in the siddhi console like in the below screenshot.

After running the respective queries in the editor to get the needed outputs, they were directed to MySQL database by dumping siddhi data to it. After dumping data into tables, using table querying visualizations were graphically visualized. For the visualizations Power BI was used.

1.    The total number of rides over the last 1 minute for each product_id.

As the first visualization the total number of rides were counted in the dataset with use of window.time() function to obtain the counts for last 1 minute.

 2.    Average taxi fare for the last 1 min for each product_id. Make sure that you have considered the rides in between the same source and destination.

This graph was complex as the final output gave a huge number of data as it was grouped using source and destination to consider about rides between same points. Therefore a tree map was used to visualize these data.

3.    Find out the highest taxi fare for the last 1 min for each cab_type.

Using window.time(1 min) inbuilt function, maximum fare was taken by grouping against cab type. The final result was visualized using a graph to compare the values easily.

 4. List down the top 3 taxi fares of each minute for each cab_type.

When calculating the top 3 taxi fares we have to be careful as streaming analytics is unlike batch analytics. In batch analytics since data is already loaded there is nothing much to be worried about. But as real-time analytics gets updated almost every second, analyzing should be done with care.

Therefore to obtain the top 3 taxi fares using the ‘limit’ function was not enough in this case. The current time and an attribute of id was used to determine this result. The resulting table is as below.

  As shown in the table the data can be sorted in the descending order using both id and time. This will retrieve he latest records in spite of traditional limit queries. Hence the final output was taken into a graph as below.

5.Find out the source location which has the least number of rides of each 1 minute.

To obtain this visualization also above theory of ordering using current time was used. The graph is as below.

The final dashboard for real-time analytics looks as below.

[enigmaticlearner]

Rich Internet Applications has made browsing faster

Stepping into tenth blog article today we will be taking a look at Rich Web-based applications.

1)Rich Internet Applications (RIAs) In the past days the world used traditional communication methods which involved in tedious methods.Gradually technology became more advanced and developed. Now we have modern tools which create complex applications through efficient browsing.These are rich internet applications.

Rich Internet Applications (RIAs) are web applications that have the features and functionality of desktop applications.Therefore they do not require software installation.

Rich Web-based applications are used for advance GUI development.They use Delta-communication.Therefore they provide good user experience.Hence these are used in most of the web-based applications.

Softwares like AJAX,Adobe Flash,Microsoft Silverlight,Java-Based RIAs use RIA technology.

2)Delta-Communication Technology (DC)

The reason for RIAs to function fast is Delta-Communication.This enables communication of necessary data between the client and server components.

1.Delta communication uses Simple-Pull-Delta-Communication (SPDC) as the simplest form. It is used in AJAX.Native JS support is used in client-side while server side doesn't need to use special technology.

2.Polling This is used to simulate data push.Like in SPDC client-side uses Native JS support and server-side doesn't use special technology.

3.Comet Like polling data-push can be simulated by Comet. Client-side uses Native JS support.A streaming server is used for server-side.

4.Server-Sent-Events (SSE) Is used (only) for true data-push.This is similar to Comet,but uses no client requests.

5.WebSocket (WS) is bi-directional This supports both data-pull and true data-push.

3)RIAs Architecture

The client-side controller contains the event handlers for the GUI •The events are triggered at the GUIs of the Views •Views are in the client-side, thus the events are triggered at the client-side

A more clear diagram is shown below about the architecture of delta communication

•Delta-communication further improves the performance, increases the scalability, and as a result provides higher user experience.

4)SPDC/AJAX

AJAX stands for Asynchronous JavaScript and XML. AJAX can be used to update parts of a webpage without reloading pages.

Popular applications like Gmail, Google Maps, Youtube, and Facebook tabs use AJAX.

                                  flow diagram of DC communication in AJAX

W3schools link to learn AJAX by tutorials is, https://api.jquery.com/

The following diagram shows functions of AJAX.

Below shows us an example of a AJAX code.

References [1] whatIs.com. “ Rich Internet Application (RIA)  ”. [Accessed: May 20, 2019]. [2] w3schools.com "jQuery-AJAX". [Accessed: May 20, 2019].

[enigmaticlearner]

Welcome to advanced version of JavaScript!!

Welcome to my ninth blog article!Today we will be taking a look at jQuery which is an advanced from of JavaScript.

So what is jQuery? jQuery is a JavaScript library.It has greatly simplified JavaScript programming.

Due to it's features like cross-browser compatibility,backward compatibility it has become very popular.Also it offers other features like, •Reveal GUI elements •Change content (based on users’ actions) •Change CSS •Delta-Communication (Ajax)

jQuery is not hard to use and it is easy to learn. It has two options to be downloaded.

1. Using official site.

It has to be downloaded by official jQuery website and then it can be embedded to source code using, <script src=“myCode.js”></script>

2. Using CDN Using CDN we will have to use networks with respect to normal way. So if we are using CDN,the link is, <script src=“CDNPath”></script>

and just like before we can use, <script src=“myCode.js”></script> to use it in the source code.

We can use jQuery API in ways like, jQuery(<selector>); -> jQuery(document);

$(<selector>); -> $(document);

*Here document provides access to the current page

To ensure that jQuery works as intended we have to make sure that the whole page is completed with loading before using JS/jQuery.

2) JQUERY SELECTORS

jQuery can use CSS selectors too.These selectors will allow us to manipulate HTML elements.

All selectors in jQuery start with the dollar sign and parentheses: $().

*Basic selectors of jQuery

*Child selector

This link is w3schools link with regard to jQuery selectors.. https://www.w3schools.com/jquery/jquery_selectors.asp

3) DOM,DOM NAVIGATION AND DOM OBJECTS

By HTML DOM,we can navigate the node tree using node relationships.

In DOM the entire document is a document node also every HTML element is an element node.

For more details you can refer the w3schools.com link given below,

https://www.w3schools.com/js/js_htmldom_navigation.asp

jQuery becomes the API inorder to traverse through the DOm.

4)EVENT HANDLING

So how do we initiate a button action in a jQuery form? That's where Event handlers come into use.These Event handlers can be used to trigger events.

Follow image shows where a event handler is used.

JQuery has different events like,

https://www.w3schools.com/jquery/jquery_ref_events.asp - more information on event handling in jQuery.

5) ADVANCED FEATURES OF JQUERY

To end today's article we will discuss advanced features of jQuery.

1. this keyword Like in programming languages "this" keyword will refer to the current object.

Ex: inside a event handler, "this" keyword refers to the element that the event has triggered on.

2. Inserting <p> at the end of <li>,referring to the button

3. Data attribute specification

4. CSS manipulation

5. Animation using jQuery

Summing up todays blog article we took a brief look at jQuery.

Below are the websites that I looked into get these information,I hope you also will find more interestings things on the way..

[1] Wikipedia.com. “ jQuery ”. [Accessed: May 12, 2019]. [2] W3Schools.com. “jQuery”. [Accessed: May 12 , 2019]. [3] jQuery.com “jQuery“. [Accessed: May 12 , 2019].

[enigmaticlearner]

Time for Client-side development!

In my eighth blog article today we are going to have a look at introduction to Client-side development.

As discussed in the fourth blog article,Distributed systems use client-side elements.Also we now know that client-side elements are

•Views – what users see (mainly GUIs) •Controllers – contain event handlers for the Views •Client-model – Business logic and data

We will be focusing on tools and technologies used to develop Client-side designs.

1.View development Browser-based clients’ Views has two main elements HTML CSS

HTML is used in making content while CSS is used in formatting.

*HTML-Hyper Text Markup Language This describes the structure of web pages using markup.

HTML elements are represented by tags. The following elements to perform it's functionality.

•Structural elements header, footer, nav, aside, article

•Text elements Tags are used to label content of the following. Headings – <h1> to <h6> Paragraph – <p> Line break - <br>

• Images Using <img> tag images can be inserted to web pages.

•Hyperlinks <hr> can be used to embed web links into web pages.

Other than these basic elements,HTML uses elements like •Data representational elements Data structure like Lists and tables are used for nesting purposes.

•Form elements These contain radio buttons,check boxes,buttons and so on.

*CSS-Cascading Style Sheets CSS are used to design web pages.

Things like the following can be done using CSS.. , Aligning Font and text editing. Color and background controls. Positioning.

CSS has three main selectors known, •Element selector • ID selector •Class selector

Also advanced selectors include pseudo classes and pseudo elements.

CSS can be used as, 1. Inline CSS 2. Internal CSS sheets 3. External CSS sheets

* Inline CSS Are used to include designs of a single element inside the code itself.

* Internal CSS sheets Used if a single page has to be included with a unique style.

* External CSS sheets Are coded as separate CSS file and used whenever needed.

*Media Query

This is a CSS technique introduced in CSS3. It uses @media rule when including a block of CSS properties. This happens only if a certain condition is true.

To learn more on HTML and CSS these links can be used..

https://www.w3schools.com/htmL/html_intro.asp

https://www.w3schools.com/css/css_howto.asp

Frameworks and libraries and plugins

View development uses frameworks,libraries and plugins too..

Some of the sophisticated tools are, jQuery,jQuery UI,Angular,React,Bootstrap,NodeJS and etc.

1.jQuery jQuery is a JavaScript Library.But this works as a framework too.

2.Angular This framework is operated by the Google.Front end applications are developed by this.

3.React Another JS library which became very popular within a short period of time.User interfaces can be built using this.

4.Bootstrap Out of frameworks this is very popular as it supports HTML, CSS, and JavaScript.It is free to download. The dimensions of a web page viewed in a computer can be adjusted to be viewed by a mobile phone as appropriate by reducing the screen size.

More information can be found in W3Schools website,

https://www.w3schools.com/bootstrap/

https://www.w3schools.com/jquery/

*Templates

Templates can maintain consistency across pages in the web site or applications. Most templates can be edited easily and they are free to be downloaded. Templates can be found for both server and client sides.

•Client-side (JS-based) template engines NUNJUCKS, PUG, MUSTACHE.JS, HANDLEBARS

•Server-side template engines Twig, jTwig, Thymeleaf, Apache Velocity

Like we discussed in the beginning of the article,browser-based clients has the two main components as,controllers and client and models.

And the above discussed frameworks,libraries and templates are used to develop these components.

In client-side component development tools main features like •DOM processing (dynamic content generation, change, removal) •Data processing •Data persistence •Session management (cookies) •Communicating (with server components)

* Generic client-side features •Form/data validation When data is entered to forms they must be validated importantly. JS can be used for this basically.

•Dynamic content generating/updating Once data is entered it must be generated and updated as wanted with user levels if needed.

•Some business logic (client-model) Business logic is used when front end and back end coding is done.

•Delta-Communication (AJAX, SSE, WS) Tools like AJAX is used to update web pages without reloading, requesting or receiving or sending data from a server

•Data formatting/preparation Data should be able to format as required and prepared for final use.

References

[1] Wikipedia.com. “ HTML ”. [Accessed: April 26, 2019].

[2]W3Schools.com. “CSS”. [Accessed: April 26 , 2019].

[3]W3Schools.com. “Bootstrap“. [Accessed: April 26 , 2019].

[enigmaticlearner]

“Data” is not simple as we think!

After a short period of time,hello again my friends! Today in our seventh blog article we will be talking about data controlling and few more new topics.

First lets learn what data and information is.Data is naturally unsorted things.

Data becomes information when they are sorted.That is when it becomes useful. Data can come in various formats like,

So data can can be stored, read, updated/modified, and deleted as we need to and thereby they can be organized in a useful manner.

At run time of software systems, data is stored in main memory, which is volatile Therefore data should be stored in non-volatile storage for persistence.

There are two main ways of storing data • Files •Databases

Out of these two types databases have proved to be much efficient. This is due to the advantages that are observed in databases.

Data independence –application programs are independent of the way the data is structured and stored. Efficient data access Enforcing integrity–provide capabilities to define and enforce constraints Ex: Data type for a name should be string Restricting unauthorized access Providing backup and recovery Concurrent access

There are many formats for storing data •Plain-text, XML, JSON, tables, text files, images, etc…

Digging more upto "data" related terms,lets take a brief look at the terms Database and Database Management System.

A database is a place where data is stored.More accurately a database is a collection of related data.Whereas a database management systems (DBMS) is a general-purpose software system that facilitates the processes of defining, constructing, manipulating, and sharing databases among various users and applications.

Also DBMSs are used to connect to the DB servers and manage the DBs and data in them •PHPMyAdmin •MySQL Workbench

In databases data can be arranged in the following manners.. •Un-structured Semi-structured data is data that has not been organized into a specialized repository, such as a database, but that nevertheless has associated information, such as metadata, that makes it more amenable to processing than raw data.

•Semi-structured Structured data is data that has been organized into a formatted repository, typically a database, so that its elements can be made addressable for more effective processing and analysis.

•Structured Unstructured data is information, in many different forms, that doesn't hew to conventional data models and thus typically isn't a good fit for a mainstream relational database.

*SQL-Structered Query Language

SQL-Structered Query Language is used to process data in a databases. Furthermore SQL can be categorized as DDL and DML.

DDL-Data definition language CRUD databases

DML-Data manipulation language CRUD data in databases

•Hierarchical databases •Network databases •Relational databases •Non-relational databases (NoSQL) •Object-oriented databases •Graph databases •Document databases are the types of databases to be found.

** Data warehouse and Big data

Data warehouse and Big data have become two popular topics in the new world.

Data warehouse a system used for reporting and data analysis, and is considered a core component of business intelligence.

Big data is a field that treats ways to analyze, systematically extract information from, or otherwise deal with data sets that are too large or complex to be dealt with by traditional data-processing application software. Big data was originally associated with three key concepts: volume, variety, and velocity.

So how do we use databases in day to day life? To process data in DB we use, •SQL statements •Prepared statements •Callable statements

                                               Connection statement codes

                                              Prepared statement codes

3.Callable statements

CallableStatement cstmt = con.prepareCall("{call anyProcedure(?, ?, ?)}"); cstmt.execute();

Other useful objects are, •Connection

•Statement

•Reader

•Result set

**ORM The mapping of relational objects (ORM, O / RM and O / R) in computer science is a programming technique for converting data between incompatible writing systems using object-oriented programming languages.

ORM implementations in JAVA • Java Beans • JPA

POJO stands for Plain Old Java Object, and would be used to describe the same things as a "Normal Class" whereas a JavaBean follows a set of rules. Most commonly Beans use getters and setters to protect their member variables, which are typically set to private and have a no-argument public constructor.

POJO stands for Plain Old Java Object, and would be used to describe the same things as a "Normal Class" whereas a JavaBean follows a set of rules. Most commonly Beans use getters and setters to protect their member variables, which are typically set to private and have a no-argument public constructor.

Beans Beans are special type of Pojos. There are some restrictions on POJO to be a bean.

POJO Vs Beans

All JavaBeans are POJOs but not all POJOs are JavaBeans. A JavaBean is a Java object that satisfies certain programming conventions: the JavaBean class must implement either Serializable or Externalizable; ... all JavaBean properties must have public setter and getter methods (as appropriate).

* Java Persistence API (JPA) Java Persistence API is a collection of classes and methods to persistently store the vast amounts of data into a database which is provided by the Oracle Corporation.

JPA can be used to reduce the burden of writing codes for relational object management. A programmer follows the ‘JPA Provider’ framework, which allows easy interaction with database instance. Here the required framework is taken over by JPA.

JPA is an open source API.Some of the products are, Eclipselink,Toplink,Hibernate.

**NoSQL SQL databases are commonly known as Relational databases(RDBMs),while NoSQL databases are called non-relational databases or distributed databases.

NoSQL comes in to need when semi-structured and un-structured data are needed to be processed.

It is advantageous to use NoSQL databases as they have high performance, supports both semi-structured and un-structured data,scalability.

MongoDB, BigTable, Redis, RavenDB Cassandra, HBase, Neo4j and CouchDB are examples of NoSQL databases.

You can find more by this link..

https://searchdatamanagement.techtarget.com/definition/NoSQL-Not-Only-SQL

**Hadoop Hadoop is an open source framework implmented by Apache.It is Java based.It is used to process large datasets across clusters of computers distributedly.Hadoop is designed to scale up from single server to thousands of machines, each offering local computation and storage.

Hadoop is consisted of two major layers,

   1.Processing/Computation layer (MapReduce), and    2.Storage layer (Hadoop Distributed File System).

Finally we'll turn into the topic Information retrieval (IR).

This is the activity of obtaining information system resources relevant to an information need from a collection. Searches can be based on full-text or other content-based indexing.

For better results, IR should have the following characteristics. 1. Fast/performance 2. Scalablitiy 3. Efficient 4. Reliable/Correct

References

[1] Wikipedia.com. “ Hadoop ”. [Accessed: April 10, 2019].

[2] TutorialsPoint.com. “JPA”. [Accessed: April 10 , 2019]

[3] TutorialsPoint.com. “Hadoop”. [Accessed: April 10 , 2019].

[enigmaticlearner]

Time to take a look at REST API!

In our last topic we talked about SOAP web service.Today we will be digging into another web service which is called REST.

REST stands for Representational State Transfer.It is advance than SOAP web service and is used mostly in current computing world.

Less bandwidth,language and platform independence and ability to use SOAP in it's implementation are advantages of REST over SOAP.

The below diagram shows differences between REST and SOAP..

                       Comparison between REST and SOAP.

                                      ------JavatPoint image-------

REST is not only used in distributed systems.It can be used in web too. This is used as a basis for HTTP 1.1 development

1)REST components

Basically REST has three major components. Namely, •Resource-based •Representations •Six Constraints

*Resource-based

HTTP Verbs like GET,POST,PUT,DELETE are used to operate in resources. Just like the names suggest using the verbs we can get the following things done..

                                          ------Wikipedia image-------

REST considers few facts when it comes to using resources. 1.Resource based 2.Representations 3.Constraints

• Resource based

1.Nouns over verbs

REST doesn't use Verbs.It uses nouns instead of them when calling URLs. ( URL-Uniform Resources Locator is used locate the resources by describing its primary access mechanism. )

EX: /Cars

But, /getAllCars is wrong to use in REST.

2 . Identified by URIs URL and URN-(Uniform Resources Name) are types of Uniform Resources Identifier-URI.

URN refer to both under the urn scheme.It remains globally unique.

3 . Multiple URIs may refer to same resource But the URL will be different according to the HTTP verb used.

Ex:One URL will contain a verb like GET but the same URL will contain a verb like POST.

Normally Request-Response model is composed by, * Request = URL + HTTP Verbs (GET,POST,PUT,DELETE)

* Response = Information in format of HTML, XML, JSON, TXT, etc…

4 . Ability to have different formats Resources can have the content in format like XML,JSON or TXT.

• Representations By this resources get manipulated and presented.

Here client and server are not usual user and service.It can have the forms like,

Ex:two web services communicating   router and a proxy communicating.

Then anyone who is requesting is the client and the server is the one that responds.

• Constraints REST has introduced 6 main constraints.

1.Client-server REST works with client-server architecture.

2.Layered system A client cannot tell whether it is connected directly to the end server, or to an intermediary along the way.These intermediate servers will make the processes work fast.

3.Stateless One client can send multiple requests to the server where each request is independent. Client holds the session information.Also layers can enforce security policies.

4.Caching As many users can send requests to the same server often they may request the same resources.Therefore it will cost resources. To avoid this unnecessary resource wastage the responses might be cached, avoiding unnecessary processing.

5.Code-On-Demand Unlike other constraints this is optional to have.REST allows client functionality to be extended by downloading and executing code in the form of applets or scripts.

6.Uniform Interface Defines the interface between clients and servers to engage. It simplifies the architecture

REST has been defined by four interface constraints: • identification of resources • manipulation of resources through representations • self-descriptive messages • hypermedia as the engine of application state

For more details you can refer these links. https://restfulapi.net/ https://en.wikipedia.org/wiki/Representational_state_transfer

2)Elements of REST style

Now lets discuss a bit about elements of REST style.REST is comprised of three types of elements,

1.Components These are programs that interact with each other. Communicates by transferring resource representations through a standard interface instead of working directly there in the same source.

2.Connector Connectors are abstract interfaces for component communication. It hides the internal implementation of resources and communications Mechanisms. This also represents the activities involved in access Resources and transfer of representations.

3.Data The data part has data elements which helps in communication.REST manages data using the following ways. •Sending a fixed-format image to the recipient •Encapsulating data •Sending raw data to recipient.

3) URLs in RESTful services

Next URLs in RESTful services is an interesting topic to talk. URL is the primary technique in RESTful communication.

• The resource a is determined by the URL segments,

While,

• The CRUD operations are determined by the relevant HTTP verb

Consider a moment where you want to retrieve data in different ways like as a collection(set of data) or as a single item or multiple items. REST has implemented methods to get data in such ways as follows.

Ex:This is considered in Read operation where data is read.URLs can be designed as in the following format.

It is advised to use nouns in plural form and spaces are not allowed in URLs.

Lastly we'll see how MVC-Model View Controller has connections with REST and what RESTful web services are..

Similar to basic MVC architecture,REST too uses it.

Apart from that there is Spring MVC which is a advanced MVC.

RESTful web services use JAX-RS API for its implementation.A JAX-RS file will have java,xml,html files.This too includes annotations which are called "JAX-RS annotations".

                                     ------JavatPoint image-------

When JAX-RS is used with annotations codes in Java will look this https://www.javatpoint.com/jax-rs-annotations-example

@GET and @Produces are web annotations used here.

References

[1] Wikipedia.com. “ Representational state transfer ”. [Accessed: March 18, 2019].

[2] JavatPoint.com. “JAX-WS Tutorial”. [Accessed: March 18, 2019].

[enigmaticlearner]

SOAP is a revolutionary topic!But what is SOAP?

Up-to my fourth blog article we have been discussing about frameworks and distributed systems.

And today lets take a view at server side development.As the first topic of server side development we'll be talking about SOAP,a famous web service technique used nowadays.

1)Web Services

So first of all what are web services and why do we need them?

Web services are a type of applications that is been developed in server side which provide a set of services.These services will be exposed via an API.(API-application programming interface-these include what to be used in a class)

Now that as we know what a web service is lets see how web services work.

XML language is used to encode all communication to a web service. Ex:When a client invokes a web service by sending a XML message,then waits for a corresponding XML response.All the communication is in XML, therefore web services aren't tied to any operating system or programming language-Java can talk with Perl;windows applications can talk with Unix applications.

Web services are built on top of TCP/IP,HTTP,HTML,Java and XML. Multiple types of clients have the ability to communicate with web service via the API. EX:Browser based clients,non-browser clients like mobile apps, POS systems and etc.

Basically, web services have the platform of XML+HTTP.All standard web services work using following components. 1.SOAP - Simple Object Access Protocol 2.UDDI - Universal Description,Discovery and Integration 3.WSDL - Web Services Description Language

Of these we will be concentrating on SOAP and WSDL for now..

Web services have some differences when compared with web applications.

2)WSDL - Web Services Description Language WSDL describes web services.It is also written in XML. It specifies the following 1.What a service does - methods 2.How a service is accessed - operations 3.Where a service is located - address(URL) is located

But why is WSDL important to us? A document written in WSDL is called a WSDL document.WSDL is simply a contract between SOAP and the client who wishes to use service. Therefore it acts as a middle user between SOAP and the client.

Mainly WSDL consists of two parts as, 1.Abstract interface section Defines signatures of procedures.

2.Deployment section Defines service location.

The most important points to be focused are on WSDL elements. So lets take a brief look at them.

This is a WSDL document,looking a bit complex but once you go through the terminologies it is not hard to understand the things.I will be adding links to refer on for more knowledge my friends.

A WSDL document has elements like <type>,<message>,<portType>, <binding>,<service>

1.<type> Declaration of data types is done here.

2.<message> The message content to be communicated is included in <message> element.

3.<portType> Includes a set of operations that has input,output and fault messages and parameter order.

In Java the equivalences for portType and operation are as follows, portType->Java interface operation->method name

4.<binding> Protocols and data format specialization is defined in this element. binding element has two main attributes, 1.name-defines the name of binding 2.attribute-points to the port for the binding

5.<service> Binds web service to a specific network-addressable location.

You can visit W3schools link to find more details.

https://www.w3schools.com/xml/xml_wsdl.asp

3)More about SOAP Back again to SOAP.We will thoroughly look at SOAP now.

As SOAP is platform independent and as it provides a way to communicate between applications running on different operating systems with different technologies and programming languages SOAP is very popular.

It supports both message oriented and functional oriented communication.

SOAP has the elements as,

1.Envelope This is the root element,it wraps all data in it. Indicates the start and end of the message to the receiver.

2.Header Contains meta-data.Optional to have but if it is existing it should be immediate after the envelope.

3.Body Has call and responsive information with message content

4.Fault Contains error and fault status. Can appear only once in the SOAP message.

5.attachmentPart SOAP messages may have one or more attachments. Each attachmentPart object has a MIME header to indicate the type of data it contains.

(MIME-Multipurpose Internet Mail Extensions is an Internet standard that extends the format of email to support Text in character sets other than ASCII)

4)Using web-services

Web services can be used through JAX-WS. JAX-WS stands for Java API for XML Web Services.This is a java library/package for developing web services.

JAX-WS uses web annotations in coding.

Java Annotation is a tag that represents the metadata i.e. attached with class, interface, methods or fields to indicate some additional information which can be used by java compiler and JVM.

These are usually written by the notation,

@webAnnotation.

Ex:

•@WebService •@WebMethod •@OneWay

5)Creating a JAX-WS web service project by Eclipse

To test our knowledge and learn more about about web services we can run a web services project using the Eclipse IDE.

A normal Java project in Eclipse can later function as a web services project.

Two separate Java projects have to be created for the client and services. The relevant Java packages can be imported by using a command like, Ex: import javax.*; The * symbol represents for importing all into the project by the relevant libraries.

References

[1] Wikipedia.com. “SOAP”. [Accessed: March 12, 2019].

[2]W3Schools.com. “XML SOAP”. [Accessed: March 12, 2019].

[3]Tutorials Point.com. “Web Services”. [Accessed: March 12, 2019].

[enigmaticlearner]

Networking is important. It’s a need to be connected worldwide!

Starting up with our fourth week,today we will be focusing on Distributed systems which has become a rising topic in the present day.

We all know from the very beginning the computer world used desktop applications which are also known as standalone applications.

These were very simple to use but when the world starting developing rapidly the need for distributed computing started rising up.More communication with other networks needed.Also occurrence of deadlocks(a deadlock is where computer processes gets locked and neither one can work until one releases the other..) in desktop applications became a headache to developers.These facts made that the world needed a new way of communication.

So lets start with distributed system and distributed computing. These two terms have a very close connection with them.Often these two terms has caused havoc and debated in computing community too.

A Distributed system consists of computers linked together through a network which allows resources to be shared.Distributed computing allows communication between the components of this distributed system.

We will now look at distributed systems Vs standalone applications

Distributed systems have proved to be very efficient since, *they have increased computing speed *works with low cost being reliable *run on multiple systems simultaneously *Doesn't affect the whole system if one component crashes.

The following are the elements of a distributed computer system.. 1.Processing components 2.Data networks for computers to communicate. 3.Data stores 4.Configuration of above elements.

Distributed systems uses protocols(rules applicable in computing world for networking mostly) For mail services which operate with the protocols like SMTP, POP3,IMAP Games and multimedia techniques uses protocols like RTP,SIP.

After the evolution of websites,web apps,web services we have now a new type of web based application which are called RIAs which are known as Rich Web Applications. So what are exactly RIAs?

RIAs are also a type of web based system which uses distributed computing. Though RIAs use distributed computing they have desktop application characteristics too.They offer us better visual experience with fast performance. These offer the user direct interaction with page elements through editing,dragging and dropping tools. Over traditional HTML pages RIAs have the best use as they do not take much time to load like HTML pages once they are refreshed or reloaded. AJAX,Adobe Flash Player,Microsoft Silverlight are popular tools which use RIA technology.

We can also find types of distributed systems which are mostly commonly used terms in the information technology field.

1.Tier architectures 2.Monolithic and micro services 3.MVC architectures (model control view architecture)

Of theses I think that most of my beloved readers have heard about tier architectures. If not lets have a brief look at them!

Starting the with the basic architecture which is client server architecture also famously known as Two tier architecture we will go up-to MVC architectures.

This table will provide you with basic features of the tier architectures.

Next comes the architectures, monolithic and micro services.

This is a comparison between monolithic and micro services architecture

MVC architecture

The MVC architecture basically has three components known, 1.Model - represents data and business layer. 2.Control - handles requests made by users. 3.View - displays data.

MVC has become a widely used software in web application development as it has advantages like ability to work in simultaneous development environments and code re-usage and so on.Originally this was introduced to desktop applications and then were converted to use in web development too. This is also used to develop user interfaces. Languages such as Java,C#,Python uses this architecture.

And of course this architecture has drawbacks too. *Increased complexity *Need of multiple programmers for the relevant layers *Requirement of knowledge of multiple technologies as MVC has many sides to coped with.

Further more MVC architecture can be categorized as Classic MVC,Client side MVC,Client and server MVC and MVC web version.

*Classic MVC

Classic MVC has the following model and it is important to note that in this model only one way communication is present between control and view.

*Client side MVC

Only client is present.Spring uses this technique.

*Client and server MVC

Both client and server uses this architecture.Also known as dual MVC architecture.But this lacks proper configuration.

*MVC web version

To end about talking distributed systems we will look into communication technologies used by these. Majorly three ways are implemented to communicate in distributed systems.

1.Functional oriented communication *RPC/RMI

2.Message oriented communication *CORBA

3.Resource oriented communication *REST

The third type of communication will be discussed separately in the near future. Basically RPC and RMI has following features..

Overall RMI is better than RPC as it gives a cleaner code.Therefore errors can be identified easily.

On the other hand CORBA facilitates communication within systems that are deployed in different platforms.This also supports many languages.

To end the biggest topic we will take a look at XML and JSON which are used in data storage. These both are similar in many aspects but have some dissimilarities too.

To learn more about XML and JSON you can visit these links..

https://www.w3schools.com/js/js_json_intro.asp https://www.w3schools.com/xml/

Tomcat server

Now leaning onto the final topic,we'll take a visit to details of a Java based open source application server called Tomcat.

Tomcat started as a servlet and now functions a fully functioned server. Tomcat can be downloaded by the Tomcat official website according to your user requirements.

This is the link to official site

https://tomcat.apache.org/download-80.cgi

To run Tomcat Java kit must be working well.To find whether this is okay we can use the windows command prompt and run the simple command %ECHO JAVA HOME%

If Java has being configured correctly it will give details about Java tools or otherwise it will say that the PATHS haven't been configured correctly.Then you can configure them using the standard way(You can have a look at my first blog article to proceed)

After downloading the executable file by Tomcat site you can run it. It will give simple options to be filled prior to it's execution. Tomcat has three ports,namely 1.Server shutdown port 2.HTTP port 3.AJP port

These three ports are connectors and they play a very crucial role once it comes to web application development using Tomcat server in deploying them in the server.These ports will be listening to requests.

HTTP port will be listening to TCP protocol while AJP port will listen to AJP(Apache JServ Protocol) Usually by default 8080 is used for server shutdown port.

Sometimes these ports can stop working as they are used by other programs.Then we can go to Services in our personal computer and stop the service accordingly.

[enigmaticlearner]

I wish if there were less errors in this code...

Hello again! In the third week,this blog article we will be discussing about code quality,dependency management and build tools.Maven will be the explicitly discussed topic.

So lets start with code quality! As the blog article title itself says we all wish to have less errors in our source codes.To have less errors here's some useful tips to start coding in a error free method as a start.

1)Code quality

Software developers start from a simple "Hello world" program and then go up to more complex codes when they become more experienced in the field.These kind of codes can contain thousand number of code lines sometimes.At such instances it's hard to focus on correcting the code if an error is identified.This is one of the very important places where "code quality" comes into use. Code quality implies mostly the maintaining quality of code.

To improve this, 1.We can add comments, by this it will help any other person to understand the code. 2.Follow coding conventions. 3.Have separate classes instead of coding a whole program in one single class. 4.Use meaningful terms when coding. Ex:use meaningful method names;for a calculation method we can code as follows Public void Calculate()

If quality of code can be maintained well the code will get less complex.Less complex the code is, more human readable it is. Also errors can be easily tracked out when comments are added at respective places. When the code is maintained well resources will not be wasted unnecessarily.These kinds of codes will have a long span and will not be thrown away once the use is over.Instead they will be used over and over again.

Code quality cannot be measured that much easily.But tools have been implemented nowadays and some of them are, *weighted micro function points *halstead complexity measures *cyclomatic complexity *lines of code *lines of code per method

Languages like Java,C,C++ has tools developed for them itself. Apart from them there are free tools and multi language support tools as well. To detect code quality in HTML there are HTML validators like CSE HTML validators are present.

As the second topic we will be moving on to dependency management and tools related to them.

2)Dependency/Package management

At olden days softwares used to work alone.But the current softwares require third party tools like libraries,third party libraries, frameworks and etc.When these external tools are cooperated with softwares problems arise later.Problems are introduced when integrating together due to different reasons. 1.Having different versions with regard to the main software. 2.Different artifacts may use different ways of integration.

As a remedy to this problem,dependency management tools were introduced.These sophisticated tools will pull things together and integrate things into a common file.

These files will be in the format of, Ex:XML,JSON,Gradle

These files will be used in finding *what dependencies to get *what version of dependency to deal with *which repository to get relevant files from

A repository will have declared dependencies.Dependencies can be fetched using name and version of the relevant dependency. Most of the dependencies have dedicated repositories for themselves. Ex:For Maven, the repository is maven central.

Therefore when a dependency is declared in a configuration file (Ex:composer.JSON) the dependency manager will access the repository to install the dependency that match the exact criteria we have set in the configuration file and make it available in the execution environment for use.

Java has Maven as its specialized tool for dependency management. PHP has Composer and JavaScript has Bower. Other such dependency management tools are Gradle,Node Package Manager.

MAVEN

Talking about Maven, Maven is not only just a dependency management tool. It has other special qualities regarding project management which has made it more specific.It functions as a multi module project management tool and has backward compatibility. Maven has the ability to download its needed files automatically into necessary directories. Developers do not have to code each and every configuration detail in the code.Only a few changes in the pom.XML file is required. Maven is used in back end while JavaScript is used in back end coding.

Maven build life cycle

A build life cycle is a well defined sequence of phases, which define the order in which the goals are to be executed. (according to Tutorials Point) Here a phase is a stage in life cycle.Process for building and distributing particular artifact is clearly defined.Therefore only a few set of commands is to be known by a person when using Maven.Lets look at Maven a little bit more.

Maven life cycle

According to Tutorials Point Maven life cycle is comprised of following phases.

     2.Build profile

Build profile consists of set of configuration values, which can be used to set or override default values of Maven build.Using a build profile we can customize build for different environments such as Production vs development environment.

    3.Build phase This comes inside the build life cycle. If phase is not explicitly defined the default phase is chosen.

Maven uses conventions over configurations;that is coders are not required to create build processes themselves.

Phases include the following, 1.Validate - checks whether configuration and code is proper. 2.test - executes test cases. 3.package - package the class files. 4.install - installs to a maven(local)repository and helps you publish jar files into a local repository. 5.deploy - publish the artifact to the remote repository.

     4.Build goals Each phase is a sequence of goals and each goal is responsible for a specific task.

Ant too is a software like Maven,but since Maven has a build life cycle it's dependency and project management it has proved to work much better.Maven requires only small changes to the configuration.

Maven is available in Eclipse IDE too.

3)Build tools Build tools automates the process of compiling computer source code into binary code.Basically build automation is the act of scripting or automating a wide variety of tasks that software developers do, 1.downloading dependencies 2.compiling source code into binary code 3.packaging that binary code 4.running tasks 5.deployment into production systems

*Use of build tools In small projects developers can manually invoke build processes.But when this comes to larger scale projects this is not practical.When the project becomes large its hard to keep track of what needs to be built,in what sequence and what dependencies are needed in building processes.Using an automation tool,it allows the build process to be more consistent.

Ex:Ant,Maven,Gradle,N Ant,Ms Build

-Build automation types

1.On-demand automation

2.Scheduled automation

3.Triggered automation

-Build automation softwares

4)Other tools used in software coding

As the end of this week’s article four more topics related to coding community will be discussed briefly.

Dev Ops

Dev ops combines software development with information technology operations. This has an agile relationship between development and information technology operations. The goal is to improve the relationship by advocating better communication and collaboration between these two business units.

Continuous integration

This is the development practice that requires coders to integrate code into a repository several times a day. Therefore problems can be detected soon and often.

Configuration management

This is a system engineering process for establishing and maintaining consistency of a product's performance along it's physical attributes throughout it's life cycle. This process is widely used in military field.

Bug tracking tools

These tools are equipped in detecting errors.

Ex:Plutora,software testing,Backlog,Bugzilla

Agile method

This practice is used in most team projects in current days.This has increased team work in team projects.Agile method has proved to be very efficient in many ways.It interacts with the client quite a lot.

Marking the end of another week’s blog article by this.Lets meet again soon with more new and interesting topics.Till then goodbye!

[enigmaticlearner]

Not to worry anymore! You can now code and sort problems TOGETHER!

The second blog article written by me will contain information on version control systems,centralized distributed systems, virtualization and about Git which is a very interesting tool used by software developing community.

Let’s start with version control systems which is known by VCs in short form.

1.Version control systems

Nowadays working in a group basis has proved be very efficient in every field.When several coders get together and start coding a group project they get to face many difficulties.One member's code will be needed by another member but they might be unable to share the code at some point.Also if the source code go wrong it would create bugs in the code and will even cause uneasiness among team members.

These kind of reasons made Version control systems to attain its birth.

Version control system is a type of tool used in group projects mostly.These help a coding team to manage changes over a source code over time.VCs can keep track on modification done to codes.Hence if a mistake is made, coders can take a look at the early versions and modify them easily.

Now we will take a view at types of VCs.

Three major types of VCs can be found in programming world.

1.Local VCs

2.Centralized VCs

3.Distributed VCs

 * Local VCs

This is the oldest type of VCs found.No collaboration is present in Local VCs.Highly prone to risks.These use a local database.

*Centralized VCs

Better than local VCs.Comprised of a centralized server which tracks file changing.Collaboration is present.Risks are server crash down and security problems.

 *Distributed VCs

Distributed VCs are the tools that are widely used in 21st century and out of them the most popular one is Git. Even though there can be a server crash down, since data is distributed,every machine contains a copy of source code with its history.Hence files could be obtained from other local machines.

Moving onto Git back again,it is popular over other distributed VCs as,

*It is a free and open source software.

*Has a prominent speed

*Takes snapshots from time to time

*Codes can't be changed without being alerted.

*Performs automatic garbage collection.

 Git has several implementations like GitHub,Git Lab,Sourceforge.

 If we consider GitHub,

GitHub is the remote repository whereas the local repository is your machine.A repository (also called a repo) is where the source code and files are stored.

Mostly using Git,project is copied from remote repository to local repository.It can happen vice-versa too.In GitHub Git Bash is used for these purposes.Git Bash is like command prompt in windows.

Lets assume that we are using GitHub.

If we consider from remote repository side, What we need to do first is creating an account in GitHub.This is free.After that we can create a repository.This remote repository can be cloned into the local machine using the command "git clone" followed by entering the link of remote repo.

Ex: git clone https://....

After this we can modify the files and commit changes. The command "git commit" will record changes into the local repo. When using the commit command it is preferred to add comments along the command to provide help when detecting errors.

If the needed commits are made we can push the file into the repo.

                Image from Tutorialspoint showing life cycle of git operations

If the repo is made in the local machine the repo will not become a git repo until you initialize it using the command "git init". Further more Git has now provided its service as egit in Eclipse IDE too.

2)Centralized delivery networks (CDNs)

This is a type of server system working with the help of neural networks.It delivers web content based on geographical locations of the user. Over traditional web hosting servers CDNs have shown that they are much effective.

  The advantages are,

*Speeds up access.

*Takes less time for downloads.The closer the CDN server to the relevant web server,the faster the loading.

*Is reliable.

*Bandwidth is saved with web host.

*Has high standard of authentication and encryption.

There are types of CDNs like

*Free CDNs

*Telco CDNs

*Multi CDNs

 Free CDNs contain CloudFare and Playster while commercial CDNs have softwares like BitTorrent,RawFlow.

 Lastly let’s have a brief look at another trending topic,

4.Virtualization

The programming world is no more focusing on hardware space. Rather, the attention has diverted towards storing data in virtual space.This is called "virtualization".

 The best example is where you can find a multiple number of operating systems running on a single hardware platform.

The key component functioning in a virtual machine is called a "hypervisor". Virtual machine monitor also is a synonym for hypervisor.It makes the virtual machine to run. Hypervisor manages the virtual operating platform making guest operating systems run on host machines.

Enhanced performance,solution to lack of storage space and limited use of resources are advantages of using virtual machines.

 *Virtualization types

1.Hardware virtualization

Ex:VMs

 2.OS level virtualization

Ex:Remote desktop terminals

 3.Application level virtualization

Ex:Runtime environments

4.Containers/dockers

 Out of above types, Containers are special due to a specific reason.

These are similar to virtual machines but these do not contain guest operating systems.Therefore these function efficiently over virtual machines as less resources are used.

 Virtualization tools

Packer

      2.VirtualBox

      3.KVM

 End of another week,start of another more set of blog articles my beloved friends!

Await for more interesting news..Till then have a nice day all!! :

[enigmaticlearner]

It’s all about PROGRAMMING!!

Computer programming has become a very popular trend in the 21st century since information technology plays a vital role in day to day life.

Today I'm going to discuss on the topic programming paradigms, Programming frameworks, Java; a very useful programming language.

 A programming paradigm is a way of programming.

Some popular programming paradigms are

1.Non structured programming

2.Event-driven programming

3.Structured programming also known as functional programming

4.Procedural programming

Further more structured programming can be more broken

down into as

functional programming and object oriented programming.

Commonly structured programming has control structures,

functions,methods and classes and so on.

 **Functional Programming

Lets discuss a bit about functional programming.

It has characteristics like,

*Its outputs of functions depend on inputs on functions.

*The global state of a system doesn't affect result of a function.

*Execution of a function doesn't affect global state of the system.

*It uses declarative approach.

Here declarative approach describe the execution of the code.

Has no side effects.In computer programming no side effects means

execution of a program doesn't affect the global state of the system.

In the other way around imperative approach is used by procedural

programming and it explains the flow of the execution.

Has side effects.

The following table compares imperative vs declarative.

 A special topic in functional programming is Lambda Calculus.

This is a mathematical theory related to mathematical functions.

It is one of the very basics of functional programming.

It was found by Alonzo Church around 1930s.

 **Event driven Programming

Speaking about event driven programming they tend to work with hardware

and messages and sensors.

Event driven programming is much different from non structured and

structured programming.

The flow of program is decided by user actions,sensors,messages.

 **Object Oriented Programming

Object oriented programming is very popular nowadays as it has key

features like abstraction,encapsulation,polymorphism,

inheritance.

 Abstraction can be explained by the aid of an example like,

although we see the mobile phone as a simple device

its inside is comprised of many complex parts.Likewise abstraction

will close the complex parts(in programming,the code) from

the outer world but yet function simply in the outer world.

 Polymorphism as the term itself means 'having many forms'. It will allow

an interface to be used to several times for a class of actions.

Encapsulation will bind methods and attributes together.

It will avoid misuse of code and keep it safe.this can achieved by access

modifiers like public and private.

By inheritance a child class/method can use parent class/method without re

declaring itself that is directly using it's parent class/method.

**Types of languages according to execution and compilation

Lets move on to types of languages in which they can be categorized into

according to execution and compilation..

 **Virtual Machines

Nowadays virtual machines are used a lot in the industry of information

technology.

They work inside a physical system and act almost like a physical

environment without letting user feel that it is virtual.

They have the ability to run multiple operating systems in one physical machine.

Division of resources between virtual machines can be done by them.

Ex:VMware workstation

·       Development tools in software

CASE stands for computer aided software engineering.

It is involved with the software engineering life cycle.

 Also three types of CASE software types can be found as,

CASE tools,CASE workbenches,CASE environments.

  ** Frameworks,Plugins and Libraries

The above three components play a very important role

in the software development area.

 Plugins

Plugin= a package of (source codes+modules+packages+executable files)

This is additional to the existing software.

Ex:Adobe Flash Player

 Libraries

Provides an application programming interface also known as API

contains configuration data,classes,data and values

Is called by the code at the run time.

 Framework

This is a huge collection compared to a plugin or a library.

A framework contains libraries,tools,rules,structures.

Not like in a library, the program's flow of control is in hands of the framework but not in hands of the caller.

The framework is reusable.It also can be extended.

Ex: .net,ActiveX

  **Java

To mark the end of this week's blog article,let's discuss a bit about

Java,one of the most powerful programming languages that has been invented.

Java was invented by James Gosling and originally was called as "Oak". As a tribute to the large amounts of coffee which was consumed by the programming team who was designing the language later it was named as Java.

Unlike other languages the biggest advantage of Java is its platform independence.

Due to this feature Java can run on any OS.Also java is an object oriented language.

JVM,JRE and JDK are three main components that are important for running Java.

The acronyms stands for the words as follows.

JAM-java virtual machine

JRE-java run time environment

JDK-java development kit

 JDK is comprised of JRE and development tools.JDK

physically exists.JRE provides the run time environment

for Java to run.The JVM exists inside the JRE.

When Java is installed to the computer we have

two foremost things.

1.Set PATH variable

2.Set JAVA_HOME variable

Both these variables are environment variables.

Most commonly environment variables contain information

like drive,path,file name,etc.

The JAVA_HOME environment variable points to the directory where JRE is installed on your computer.The intention is to point to where Java is installed.

The PATH variable has to be set when we get to use tools such as javac,java and etc.

We can set PATH in two ways;

1.Temporary

2.Permanent

If we are setting the variable temporarily,we can use the windows command prompt.But since this is tiresome,we can use the advanced settings to set the PATH permanently.

Java has many IDE s right now.

Eclipse,IntelliJ,Net Beans,Android Studio are some of them, where Android Studio is widely used for developing mobile apps.

Lets meet in next week with more interesting news!

[enigmaticlearner]

First ever blog

Hey there!I’m back..