jguertl

[jguertl]

Attributed Face Generation using DCGANs

With the rapidly increasing amount of images, the automatic generation of large image collections has been actively pursued in a number of computer vision domains. However, the creation of realistic face images has been limited. In this post, I am going to show how to generate human faces using Deep Convolutional Generative Adversarial Network (DCGAN), which probably do not exist in real life. 

DCGANs are a class of neural networks that have gained popularity recently. They have been designed to use convolutional layers in both the discriminator and generator. Recently, stable DCGANs were able to generate realistic images in several domains. Based on this, we extend the network to accept face attributes as additional inputs as described in the architecture above. In detail, we expand the standard DCGAN architecture by inserting conditional information during training and inference of the networks. Note that we encode the attributes (e.g. Male, Blonde,...) as vectors into both the generator and discriminator.

We evaluate the performance of our method on the CelebA dataset. It is a large-scale public dataset with more than 200.000 celebrity images.

In terms of exploring conditional attributes, we are interested in how the additional condition might be used to deterministically specify the faces generated by the network. During generation, we focused on face attributes which have a clear visual effect. Rare attributes in the dataset would likely be treated as noise by the network. The two figures show side-by-side illustration of generations from the network, the describing attributes, and the inherent dataset image. We observe that the networks succeeds specifically in generation of faces based on local attributes. The code to reproduce the experiments, called CDCGAN, can be found here.

[jguertl]

Custom Dataset and DataLoader in Gluon

One of the most critical steps for model training and inference is loading the data: without data you can’t do machine learning. In this post I want to show you how to use the Gluon API to define a custom Dataset and iterate through it by using a custom DataLoader.

Dataset

Objects of type Dataset are used to represent collections of data, and include methods to load and parse the data. Gluon has a number of different Dataset classes for working with image data straight out-of-the-box, for example ArrayDataset or ImageFolderDataset. Sometimes you have data that doesn’t quite fit the format expected by the included Datasets. You might be able to preprocess your data to fit the expected format, but it is easy to create your own dataset to do this. All you need to do is create a class that implements a getitem method, that returns a sample (e.g. an image/boundingbox pair). Below you can find a custom Dataset class used in the MXNet CycleGan implementation.

DataLoader

A DataLoader is used to create mini-batches of samples from a Dataset, and provides a convenient iterator interface for looping these batches. It’s typically much more efficient to pass a mini-batch of data through a neural network than a single sample at a time, because the computation can be performed in parallel. A required parameter of DataLoader is the size of the mini-batches you want to create, called batch_size. Before Gluon’s DataLoader, MXNet used DataIter objects for loading data for training and testing. Note to take care of pre-processing steps such as augmentations in a transform function when switching to the new DataLoader.

In this example, we can see mini-batches of data (and labels), each with 5 samples. When comparing the shape of the batches to the samples returned by the Dataset, we’ve gained an extra dimension at the start which is sometimes called the batch axis. Our data_loader loop will stop when every sample of dataset has been returned as part of a batch. Sometimes the dataset length isn’t divisible by the mini-batch size, leaving a final batch with a smaller number of samples.

[jguertl]

High-Resolution Image Synthesis with Conditional GANs

The collection of a large scale dataset is a challenge and the key question in order to train robust machine learning models. Moreover, the annotation of large collections of images is both labor-intensive and error-prone. The paper High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs from Wang et. al. discusses a new approach to produce high-resolution images from semantic label maps, which has a high range of application. For example, it can reduce the time annotate a large collection of images as described before. In addition, the discussed method also gives us new tools for higher-level image editing e.g. chaning the appearance of existing objects in real time.

To generate this high quality the team extended a baseline pipeline called pix2pix with an improved objective function and a better network design. The network architecture consists of two seperate generators: a global generator and a local enhancer. After some initial setup they are trained jointly on high resolution images. On the other hand, the discriminator needs to have a larger receptive field than for normal images. To adress this issue, the proposed a multi-scale discriminator using three identical networks working on different image scales. With the discriminators, the learning problem then becomes a multi-task learning problem.

Finally, the problem of generating diverse realistic images using the same semantic label map is tackled. To generate diverse images, an additional low-dimensional feature channels as input to the generator is used. By manipulating these features, one has flexible control over the synthesis process. As a result, several images generated out of the same label map can be generated.

[jguertl]

Improving the Realism of Synthetic Images

To achieve high accuracy in training of neural nets, the training sets need to be large, diverse, and accurately annotated, which is usually very costly. Therefore it has become more popular to train models on synthetic images, avoiding the need for expensive human annotations. However, they may not generalize and end up in poor network performances. To close this gap, this paper proposes a method for refining synthetic images to make them look more realistic.

The realism of a simulator’s output gets improved by using an adversarial network similar to Generative Adversarial Networks, but with synthetic images as inputs instead of random vectors. Additionally, to stabilize training and avoid artificats, some key modifications to the standard GAN are made:

‘self-regularization’ term

a local adversarial loss

updating the discriminator using a history of refined images

This enables generation of highly realistic images from synthetic data, which they demonstrate both qualitatively and with a user study.

[jguertl]

Vector arithmetic on GAN face samples

As described in a previous post, GANs have two networks, a generator and a discriminator. Images get generated by the generator, who upsamples a random vector noise from a simple distribution. The vector usually mostly consinsts of different features in a noisy sort. In the example below the vector encodes 40 random values, representing the 40 different attributes of the celebA dataset.

These vector characteristics allow us to easy manipulate the semantic qualities of many generated samples using arithmetic operations. Therefore we consider the noise after looking into the output images and try to do simple operations with them. Experiments have shown that the average of at least images is needed to get reasonable results. As demonstrated in the image, the calculation of vector("smiling female") - vector("neutral female") + vector("neutral male") produces a vector("smiling male") after running the generator.

This technique can be also be used to linearly model the vector representation, e.g. manipulate a person from male to female. Further work on the above mentioned methods could dramatically reduce the amount of labeled data needed for attributed generation of images.

[jguertl]

Conditional CycleGAN for attributed super-resolution

The most recent development in Generative Adversarial Network (GAN) such as cycleGAN present an approach for learning to translate an image from a source domain to a target domain without paired examples. These cycleGAN can be extended to conditional cycleGAN so that the mapping betwenn domains is subjected to an attribute condition. Using single-image super-resolution as an application exmaple, a set of facial appearance (e.g. gender, glasses, skin) can be incorporated into the network by an additional layer. The link shows sample results for attribute guided face generation. Apart from label-defined changes, the generated high resolution share common properties with the input image, such as orientation and postition of the person. Note that some generated results for labels like 'bald' may not be totally convincing for these examples. To summarize, in the attribute guided conditional cycleGAN the adversarial loss is adapted to include a conditional feature vector as parts of the generator and discriminator network. While the application of this paper focuses on faces, the conditional cycleGAN is general and can be easily extended to other applications.

[jguertl]

Generative Adversarial Networks (GANs)

Recently there has been a large comeback of interest in generative models, especially within the deep learning community. Generative adversarial networks (GANs) have extended the possibilities to create computer-generated content, and have shown active research since this concept was introduced by Ian Goodfellow in 2015. The networks try to create synthetic data similar to some known real input data. For example, researchers have created interactive 3D models and Text-to-Image synthesis using this technique.

There are two main elements of a GAN - a Generator Neural Network and a Discriminator Neural Network. The Generator Network acquires a random input noise and tries to create a sample of data. The Discriminator Network takes input either from the real world examples or from the generator. It is then responsible to decide whether the input is real or generated. This is done by a binary classification giving output in the range 0 to 1.

Now the training of the whole adversarial network is done as a steady contest between the generator and discriminator. Both parts can be optimized independently. This method is taken from game theory called two-player minimax game. Under certain conditions, the training process reaches a point where the discriminator cannot classify generated examples from real ones.

[jguertl]

TensorFlow real-time recognition on iOS

TensorFlow, Google's open source library for machine learning, is available for Apple's iOS mobile platform since this year. To build TensorFlow on iOS, Google has provided a set of scripts, including a makefile, to drive the cross-compilation process. Let us start by installing TensorFlow, you just have to follow the installation instructions.

This creates a static library containing the core code that you can link any Xcode project against. Because the iOS app does not come with the inception model we will need to download it and copy it across to the data directory. You should now be able to run the live camera app on your device.

[jguertl]

Upload images using Django REST and AngularJS

Handling image uploads via AngularJS with Django is a little less than straightforward. The best way to do so is to fetch the image as a Bas64 encoded string and push that up, rather than the image file itself. Base64 is a way to encode and decode binary data so that it can be transmitted more easily.

The client-side image upload is done by a simple HTML input field. After submitting, an AngularJS function which reads the image source is called. During the upload the image is stored inside a JSON Object.

On the server-side we need a Django REST framework field for handling image-uploads through raw post data. The easiest way is to use the dfr-extra-fields Base64ImageField, which serializes a Base64 image to a string. Just import the field and use it inside your serializer.

Finally we want to show the images uploaded by the user. During development, you can serve user-uploaded media files from MEDIA_ROOT using the django.contrib.staticfiles.views.serve() view. Remember that this is not suitable for production use and requires Django 1.9 or newer.

[jguertl]

Share Plugin for Xamarin and Windows

Xamarin plugins are a special kind of NuGet package that let you easily add cross-platform functionality to your apps. James Montemagno and I have been slowly adding new features and have just published a new version of the plugin to share text, links, or open a browser. In addition to refactoring the code we added support for several platforms and changed the package name. This introduced a few breaking changes and therefore we decided to make the new version a major update.

Release Notes & New Features (Version 3.0.1)

New namespace and package name

Add open browser to url

Add WinRT support

Remove dependency on Xamarin.Forms

iOS unified support

Bug fixes

Head over to NuGet to update to the latest version. Examples for using the plugin on different platforms can be found at the GitHub repository.

[jguertl]

Worthy F# libraries

Now that the F# community is getting more attention it is just a good time for looking into F# open source libraries. With its open development process, F# has a strong ecosystem of frameworks that developers can freely use in their own applications. While some are for very specific uses that you may never need, others are so helpful that you should never start without them.

__Suave__ Suave is a lightweight, non-blocking web server which is efficient and suitable for building fast and scalable network applications. It comes with support for HTTPS, multiple TCP/IP bindings, Basic Access Authentication and more. Below is all you need to start a web server on default port 8083 over HTTP.

__FAKE__ F# Make (Fake) is a build-and-publish automation utility. It is using an easy domain-specific language (DSL) so that you can even start using it without learning F#. To get started with a FAKE script, the general idea is that you would do something like this:

This build script has two targets. The "Deploy" target has exactly one dependency, namely the "Test" target. Invoking the "Deploy" target will cause FAKE to invoke the "Test" target as well.

__Paket__ On of my favorite tools! Paket is another way to manage dependencies in your .NET projects. It essentially just takes the functionality of NuGet and adds some extra nice features.

What are the advantages of using Paket? • If two packages reference conflicting versions of a package, NuGet will silently take the latest version. Paket gives you control over this process. • Paket also enables you to reference files directly from GitHub (and Gist) repositories or any http-resource.

__FunScript__ FunScript is an F# compiler library enabling developers to write single-page web applications in F# and running them in the browser or on the server via Node.js. FunScript has two components: a compiler library used to convert F# code into JavaScript and a TypeScript type provider used to expose the APIs of various JavaScript libraries and the DOM as static types.

An example written in FunScript looks like this:

__FsCheck__ You may not need this for every app, but if you doing Test-Driven Development, then you should use FsCheck. The F# version of the excelent Haskell QuickCheck test library allows expressing properties about programs and testing them with randomly-generated values. A property for FsCheck is simply a truth-valued function whose arguments are implicitly universally quantified. Check out the documentation for some code examples.

[jguertl]

Semi Structured Data

Semi-structured data is not strictly typed, but has a certain structure, even if this is not immediately apparent. Data from relational or object-oriented databases is strictly typed, because it has to fit into the scheme of the tables or classes. All that lies between these two extremes is semi-structured data. In these, the structure may be implicit, irregular or partially. Examples of semi structured data are CSV or JSON.

Import Steps:

Normalize the data

Select one method to import the data

Perform the import

Before you import

Importing data can be a very challenging task when the migration is not well planned. The larger the system is, the harder is the data import, especially if there is existing data. The probability that INSERT operations running processes block or cause data loss is higher if there is already data inside the database. Before you import make sure to normalize the data model. That means that there will not be any redundance.

Methods of data import:

SQL INSERT: The data can easily be converted into valid INSERT statements. These are then executed. A script that inserts data into a database is called DML script. An SQL INSERT statement is probably the most frequently used data import method.

SQL Developer: SQL Developer is a program from Oracle that provides a user interface for managing databases forms. It also has an extended function for importing data which can be configured (Tools > Preferences > Migration )

SQL Loader: The SQL Loader populates Oracle tables with data from flat files. It gives a lot of flexibility to selectively load certain columns but not others, or to exclude certain records entirely.

Data Pump: The Data Pump toole from Oracle is able to migrate the objective data from an Oracle database to another. Goal during the development of Data bump was to replace old tools with a faster data exchange. There two methods are of import and export, the Direct Path Method for imports with simple structure and the External Path Method for import with triggers.

Choosing the right method:

Format: Possible formats are binary or csv files.

Amount of data: The amount of data is not only described by the number of lines, but also by the complexity of the import.

Performance: The performance of the host system also has an impact on the decision. A DBMS with a low CPU and little memory needs more time for an insert.

Platform: Errors might occur while migrating data from one database to another database management system.

Database structure: The design of the database is very important. A lot of triggers and foreign keys slow down INSERT actions.

Frequency: If you want to import data more often, you should think about creating a professional and stable workflow.

[jguertl]

Hierarchical queries in SQL

What are Hierarchies?

Hierarchies are treel-like structures that can help you understanding the organizational structure of your data. This is accomplished by super- and subordinate items. There are two types of hierarchies: the mono-hierarchy and the poly-hierarchy. In a mono-hierarchy, each item has only one item that is „above“ it, while in a poly-hierarchy, each item can have a unlimited number of items „above“ it.

You can connect hierarchical (tree) structures in Oracle by connecting rows. This connection with the predecessor/parent can be done with the CONNECT BY clause. The root is defined with the START WITH clause.

Hierarchical tree

Processing of hierarchical queries

A join, if present, is evaluated first.

The CONNECT BY condition is evaluated.

Any remaining WHERE-clause predicates are evaluated.

Formation of the hierarchy using the information from these evaluations

Oracle selects the root rows. (Those with the START WITH condition)

Oracle selects the child rows from the corresponding root row. The CONNECT BY condition must be satisfied with one of the root rows.

Oracle next selects the children of the rows returned in the step before, and then the children of them, and so on. The CONNECT BY condition is evaluated with respect to a current parent row.

If there is no join, all the rows that do not satisfy the condition of the WHERE clause are eliminated. Oracle evaluates this condition for each row individually, rather than removing all the children of a row that does not satisfy that condition.

Oracle returns the rows in the order shown below.

Examples

Returns all the paths from the root to the leafs.

Returns all the paths from one special leaf ('ADAMS') to the root.

[jguertl]

Writing a Xamarin.Forms Plugin

Xamarin.Forms is Xamarin’s cross-platform UI framework that makes it possible to build user interfaces for Android, iOS and Windows Phone using XAML. It enables you to share not only your app’s business logic, but also a shared user interface across the different platforms. A plugin for Xamarin.Forms is a kind of NuGet that adds cross-platform functionality or abstracts platform specific functionality. Usually you use a plugin in a Portable Class Library (PCL) but it can be accessed on each platform.

Setting up the project

Before you start coding make sure to install this very useful Visual Studio template. It automatically sets up the interfaces, PCLs and platform-specific projects.

After choosing the VS template "Plugin for Xamarin.Forms" the following structure will be generated:

Inside your interface you simply declare the method(s) which must be implemented by derived classes. Inside your implementation you can access native objects of platforms, for example Text-to-Speech.

Interface Implementation

Packaging

After everything is implemented you now need to package the NuGet. First you have edit the generated .nuspec file. (You can find mine here). Afterwards download this NuGet.exe and place it inside your project folder.

Next open your command line, jump into your project directory and enter the following commands. Make sure to rebuild your whole project before packaging.

As a result you should find the NuGet package in your project folder.

Publishing the plugin

Before publishing it is recommended to test the NuGet package locally in a Xamarin.Forms project. If everything goes well you are now able to externally distribute your package. Finally log in to your NuGet account and upload the file via the browser interface.

If you’d like to play around with the code from this example, you can grab it from my GitHub.

[jguertl]

Sophos UTM 9 Report

Sophos has been in the security business for a long time and you might be familiar with their antivirus software. Their UTM solution is based on Astaro technology, which they acquired around four years ago.

The abbreviation UTM stands for Unified Thread Management and is a software package provided by the company Sophos. Gartner’s Magic Quadrant now classifies Sophos as Leaders Quadrant members in the unified threat management sector. It includes features like:

Firewall → rules to protect yourself against attacks

Web Application Firewall → prevents attacks over HTTP

Mail Security → check the E-mail transfer for incoming threats

Web Security & Filtering → internet access of the local users can be filtered

Wireless-Management → if the hardware supports a wireless AP, you can set up a secured Access-point

VPN connectivity → functionality for connecting members of your office or company which are abroad into your local LAN

The appliance was tested by me using state of the art test equipment. Where possible, I mixed network traffic simulating real-world conditions. Check out my UTM 9 test report here.

[jguertl]

The n-queens problem using Xamarin.Forms

The n-queens problem consists in placing n non-attacking queens on an n-by-n chess board. A queen can attack another queen vertically, horizontally, or diagonally. The algorithm to solve this problem uses backtracking. Backtracking is similar to the brute force approach where it tries all of the solutions but the only difference is that it eliminates/avoids the partial candidate solutions as soon as it finds that that path cannot lead to a solution.

The obvious choice to implement this algorithm is a recursion. The solution is to recursively check if placing a queen at a certain column / row leads to a solution. This has been done in the class NQueenProblemSolver.

You can find the whole project here.

[jguertl]

Bitsquatting

A bit (binary unit) is how computers communicate. Computers are not perfect, and bit data can get corrupted within systems i. e. DNS resolution. These bit errors can affect the communication of a DNS request and response. Every letter of the alphabet can get corrupted in up to 8 possible ways. Not all of the resulting bit series will be alphanumeric. For example, the letter “t” can error to the number 4.

In very rare cases, someone that types in the domain name yahoo.com will end up requesting 9ahoo.com due to bit corruption. By combining lots of bitsquatted domains on some popular website names, one can collect a amount of bad traffic.

In the screenshot above I ran the domain yahoo.com against the tool urlcrazy and nearly all permutational domains are already registered. The same is true for other popular websites.

For further information check out my technical paper about bitsquatting.