Tuesday, January 16, 2018

Linear Regression - Machine Learning with TensorFlow and Oracle JET UI Explained

Machine learning topic is definitely popular these days. Some get wrong assumptions about it - they think machine could learn by itself and its kind of magic. The truth is - there is no magic, but math behind it. Machine will learn the way math model is defined for learning process. In my opinion, the best solution is a combination of machine learning math and algorithms.  Here I could relate to chatbots keeping conversational context - language processing can be done by machine learning with neural network, while intent and context processing can be executed by programmable algorithms.

If you are starting to learn machine learning - there are two essential concepts to start with:

1. Regression
2. Classification

This post is focused around regression, in the next posts I will talk about classification.

Regression is a method which calculates the best fit for a curve to summarize data. Its up to you which type of curve to choose, you should assume which type will be most suitable (this can be achieved with trial and error too) based on given data set. Regression goal is to understand data points by discovering the curve that might have generated them.

In this example I will be using simplest regression possible - linear. Line is described by equation y = W*x + b. Where b is optional and can be 0 (line will cross (0, 0) point). For complex data sets, we might use polynomial equations and generate curves.

Here is Python code which implements linear regression with TensorFlow API (I have provided comments for all steps, reading code should be self explanatory):


Key element in any kind of machine learning - cost. The higher the cost, the worse is learning output. In linear regression, cost is typically defined by the sum of errors. The error in predicting x is calculated by the squared difference between the actual value f(x) and the predicted value M(w, x). The cost is the sum of squared differences between the actual and predicted values.

As you can see in the code above, we define cost function and ask TensorFlow to run optimizer to find the optimal values for model parameters. All the hard math calculation is happening in TensorFlow, our job is to prepare training data and choose right learning approach with correct equation.

Let's run JET UI, which talks to TensorFlow through REST. Training data is randomly generated (always 100 points) during each training session.

Training Epochs - number of learning iterations during training session
Learning rate - learning speed, smaller rate - more careful learning
W - learned model parameter to calculate equation y = W*x
Cost - value which shows how successful learning was, lower cost is better

1. We start from 1 training epoch and learning rate 0.001:


Learning result is not good - red line is result of linear regression, it doesn't represent best fit for training data. Cost is pretty high too, which indicates that learning wasn't successful.

2. 10 training epochs and learning rate 0.001:


As we repeat multiple learning iterations within the same training session - learning result is better. Cost becomes smaller and linear regression calculated line fits better, but still not ideal.

3. 100 training epochs and learning rate 0.001:


It helps to increase learning iterations, cost is significantly improving and line fits much better. This means outcome for W parameter learning is pretty good.

4. 1000 training epochs and learning rate 0.001



Let's make model to learn even harder and repeat more times - cost is becoming even better.

5. 2000 training epochs and learning rate 0.001


We could increase learning iterations further, but at some point it will not help. Learning process will start to suffer from overfitting. You can think about it - learning and repeating so many times, that at the end you start forgetting things. Cost is getting worse when repeating learning iterations further.

6. 2000 training epochs and learning rate 0.0001


It should help to make learning rate smaller, which result in more careful learning. This should allow to get better learning results with higher number of learning iterations. We get best learning cost result here and the most optimal line. You may ask - what is the use of that line? It can help to predict y values which were not available in training dataset.

7. 2000 training epochs and learning rate 0.01


On contrary if we increase learning rate, learning process will be faster - optimizer will run faster. This will result in decreased model output quality, cost will be higher and W parameter value will not produce such best fit line as in previous training run.

Few hints related to Oracle JET UI. You can achieve very good data visualization with JET chart components. For example I could control marker type rendered for training data points:


Line which represent learning result, can be displayed as reference line:


To display reference line, I'm using y-axis property which comes with JET chart:


References:

- Example for Linear Regression with Python and TensorFlow - Gist
- JET UI example - GitHub
- Accessing TensorFlow model through REST - Machine Learning with Oracle JET and TensorFlow
- Book - Machine Learning with TensorFlow 

Wednesday, December 27, 2017

JET UI Shell with Dynamic Tabs Implementation

I will describe and provide sample code - how to implement dynamic tabs in Oracle JET. This is useful for module loading, default JET template comes with static tabs - this works OK for the apps with smaller number of modules. When menu structure is more complex, it makes sense to load tabs for each module dynamically, when module is selected from the menu. This allows user to focus on modules he is working on, well just same as tabs in your Web browser.


Dynamic tabs are only visual placeholders, I'm using JET router to load module and keeping tab selection in synch. There are multiple benefits for such approach - module loading is following standard JET implementation with router, visually it looks like we have modules loaded using tabs. Overall implementation for dynamic UI Shell tabs with JET becomes quite simple, when relying on JET router.

In this post I will describe how to:

1. Add new tab and select module
2. Select existing tab from the menu/by clicking on the tab itself and select module
3. Remove tab and select next available module

Dashboard module is default one, it is loaded once application is accessed and we dont allow to remove it. Other modules are accessible from menu list:


Multiple menu items are opened with tabs displayed below. Let's see how tab removal works. If I remove Incidents tab:


Next tab on right will be selected as current:


If tab is removed, which is not selected -  Customers, current tab will remain selected:


If there is no tab on the right, while removing current tab About, first tab on the left will be selected:


Implementation part - menu list is implemented in this example with JET menu component (in real application you would use more complex menu structure, may be hierarchical):


Dynamic tabs solution is based on JET tab bar component, with removable option (you can try it in JET Cookbook - Add/Remove tab example):


Application defines array of tabs and variable to to keep track of current module:


Once menu item is loaded, we check first if same was opened already. If yes - we open it, instead of opening the same tab twice. Tab array is updated with new item and JET router navigates to current module, this is how module gets displayed, no additional magic:


When user selects tab, current module is loaded through JET router:


Tab removal is handled in delete method. Here calculation is done, which tab should be focused after current tab removal. Item is removed from array of tabs and JET router state is updated:


Sample JET code is available on my GitHub repository.

Thursday, December 21, 2017

Recipe for JQuery Menu integration into ADF Faces

May be you are thinking to add some fresh looking element into your ADF app? I would propose JQuery menu. Advantage of such menu - it is purely client side implementation and it allows user to navigate through items really fast. Besides fast navigation, it looks great too and can be easily customized.

In this post I will describe how to achieve JQuery menu integration into ADF Faces and provide sample ADF application. I'm using JQuery menu from here - Animated Responsive Drop Down Menu. There is no need to implement JQuery menu component, we can use one of the existing implementations.

I have recorded short example, you can see how convenient is client side menu navigation. Each menu item loads ADF region:


JQuery menu integrated into my ADF sample:


It can display menu items for multiple levels:


Integration into ADF Faces is very simple, you need to copy JS and CSS files into ADF application:


Define JS and CSS resources in ADF main page, make sure to reference JQuery too:


Menu structure is defined in ADF fragment using HTML tags. Once menu item is selected, call to ADF is made through combination of client/server listeners (check WBClient.menuNavigation method). In my future posts I will describe how to construct similar menu structure dynamically. If your application is secured, make sure to define ADF Security permissions to TF's properly, any unprotected TF can be loaded from the menu:


Client listener method menuNavigation constructs params with target TF address and calls server listener, which in turn displays ADF dynamic region:


Menu fragment with HTML tags is integrated into ADF page template:


Download sample application - ADFJQueryMenuApp.zip.

Saturday, December 16, 2017

Machine Learning with Oracle JET and TensorFlow

Oracle JET works with any kind of REST service, such service could be the one coming from TensorFlow (read more in my previous post - TensorFlow Linear Regression Model Access with Custom REST API using Flask). I have implemented linear regression model with gradient descent optimizer in TensorFlow (will describe this in more detail in my next post - machine learning is all about mathematics) and consuming it in JET UI:


There is option to define training steps (or data points) and learning rate. As outcome we get W and b values for linear equation y = Wx + b. After training is executed (so called machine learning process) - W and b parameters are identified, this allows to predict y value for any x. More about this in my next post, today will focus on JET.

To execute training, we call TensorFlow REST API endpoint and pass number of training steps/points and learning rate. I'm have create JET helper module to perform actual REST call, to separate logic from REST call invocation. Useful technique of custom callback is used to call method located in main module from helper module. We pass callback method name REST call helper function:


REST call is executed through JQuery and success callback is assigned with our method name (instead of processing results in REST call helper method, we can do processing in main module):


This is where results processing happens - in custom callback method. We prepare x/y training dataset to be displayed in chart. We get back estimated W and b parameters, this allows to calculate estimated y based on equation y = Wx + b for all x values used in training set (to compare how well model was trained):


One more nice component of Oracle JET - Form Layout. It makes very easy to create simple forms, specify number of max columns and you will get responsive UI for form components out of the box:


Make sure to add ojformlayout module:


To debug data returned by TensorFlow REST service, check Network history in browser developer tools:


By the way - it is completely up to you to decide about REST JSON data shape. With Python jsonify from Flask you can construct JSON from arrays of data and publish your own REST end point anywhere in Python script (you code in Python for TensorFlow):


Download sample JET application (make sure to use ojet restore, before ojet serve) from GitHub. TensorFlow linear regression model used for this sample is available as GitHubGist.

Monday, December 11, 2017

TensorFlow Linear Regression Model Access with Custom REST API using Flask

In my previous post - TensorFlow - Getting Started with Docker Container and Jupyter Notebook I have described basics about how to install and run TensorFlow using Docker. Today I will describe how to give access to the machine learning model from outside of TensorFlow with REST. This is particularly useful while building JS UIs on top of TensorFlow (for example with Oracle JET).

TensorFlow supports multiple languages, but most common one is Python. I have implemented linear regression model using Python and now would like to give access to this model from the outside. For this reason I'm going to use Flask, micro-framework for Python to allow simple REST annotations directly in Python code.

To install Flask, enter into TensorFlow container:

docker exec -it RedSamuraiTensorFlowUI bash

Run Flask install:

pip install flask

Run Flask CORS install:

pip install -U flask-cors

I'm going to call REST from JS, this means TensorFlow should support CORS, otherwise request will be blocked. No worries, we can import Flask CORS support.

REST is enabled for TensorFlow model with Flask in these two lines of code:


As soon as Flask is imported and enabled we can annotate a method with REST operations and endpoint URL:


There is option to check what kind of REST operation is executed and read input parameters from POST request. This is useful to control model learning steps, for example:


If we want to collect all x/y value pairs and return in REST response, we can do that by collecting values into array:


And construct JSON response directly out of the array structure using Flask jsonify:


After we run TensorFlow model in Jupyter, it will print URL endpoint for REST service. For URL to be accessible outside TensorFlow Docker container, make sure to run TensorFlow model with 0.0.0.0 as in the screenshot below:


Here is example of TensorFlow model REST call from Postman. POST operation is executed payload and response:


All REST calls are logged in TensorFlow:


Download TensorFlow model enabled with REST from my GitHub.

Tuesday, December 5, 2017

JET Composite Component in ADF Faces UI - Deep Integration

Oracle JET team doesn't recommend or support integrating JET into ADF Faces. This post is based on my own research and doesn't reflect best practices recommended by Oracle. If you want to try the same - do it on your own risk. 

All this said, I still think finding ways of further JET integration into ADF Faces is important. Next step would be to implement editable grid JET based component and integrate it into ADF to improve fast user data entry experience.

Today post focus is around read-only JET composite component integration into ADF Faces. I would recommend to read my previous posts on similar topic, today I'm using methods described in these posts:

1. JET Composite Component - JET 4.1.0 Composite - List Item Action and Defferred Loading

2. JET and ADF integration - Improved JET Rendering in ADF

You can access source code for ADF and JET Composite Component application in my GitHub repository - jetadfcomposite.

Let's start from UI. I have implemented ADF application with regions. One of the regions contains JET Composite. There is ADF Query which sends result into JET Composite. There is integration between JET Composite and ADF - when link is clicked in JET Composite - ADF form is refreshed and it displays current row corresponding to selected item in JET Composite. List on the left is rendered from series of JET components, component implements one list item:


As you can see, there are two type calls:

1. ADF Query sends result and JET Composite. ADF -> JET call
2. JET Composite is forcing ADF form to display row data for selected item. JET -> ADF call

Very important to mention - JET Composite is getting data directly from ADF Bindings, there is no REST layer here. This simplifies JET implementation in ADF Faces significantly.

What is the advantage of using JET Composite in ADF Faces? Answer - improved client side performance. For example, this component allows to expand item. Such action in pure ADF Faces component would produce request to the server. While in JET it happens on the client, since processing is done in JS:


There is no call to the server made when item is expanded:


Out of the box - JET Composite works well with ADF Faces geometry. In this example, JET Composite is located inside ADF Panel Splitter. When Panel Splitter is resized, JET Composite UI is nicely resized too, since it is out of the box responsive. Another advantage of using JET Composite in ADF Faces UI:


When link "Open" is clicked in JET Composite - JS call is made and through ADF Server Listener we update current row in ADF to corresponding data. This shows how we can send events from JET Composite to ADF Faces:


It works to navigate to another region:


And come back - JET content is displayed fine even after ADF Faces PPR was executed (simple trick is required for this to work, see below). If we explore page source, we will see that each JET Composite element is stamped in HTML within ADF Faces HTML structure:


Great thing is - JET Composite which runs in JET, doesnt require any changes to run in ADF Faces. In my example, I only added hidden ID field value to JET Composite, to be able to pass it to ADF to set current row later:


I should give couple of hints regarding infrastructure. It is not convenient to copy JET Composite code directly into ADF application. More convenient is to wrap JET code into JAR and attach it this way to ADF. To achieve that, I would recommend to create empty Web project in JDEV, copy JET Composite code there (into public_html folder) and build JAR out of it:


Put all JET content into JAR:


If Web project is located within main ADF app, make sure to use Working Sets and filter it out to avoid it to be included into EAR during build process:


Now you can add JAR with JET into ADF app:


In order for JET HTML/JS resources to be accessible from JAR file, make sure to add required config into main ADF application web.xml file. Add ADF resources servlet, if it is not added already:


Add servlet mapping, this will allow to load content from ADF JAR library:


To load such resources as JSON, CSS, etc. from ADF JAR, add ADF library filter and list all extensions to be loaded from JAR:


Add FORWARD and REQUEST dispatcher filter mapping for ADF library filter from above:


As I mentioned above, JET Composite is rendered directly with ADF Bindings data, without calling any REST service. This simplifies JET Composite implementation in ADF Faces. It is simply rendered through ADF Faces iterator. JET Composite properties are assigned with ADF Faces EL expressions to get data from ADF Bindings:


JET is not compatible with ADF PPR request/response. If JET content is included into ADF PPR response - context gets corrupted and is not displayed anymore. To overcome this we are re-drawing JET context, if it was included into PPR response. This doesn't reload JET modules, but simply re-draws UI. In my example, ADF Query sends PPR request to the area where JET Composite renders result. I have overridden query listener:


Other methods, where PPR is generated for JET Composite - tab switch and More link, which loads more results. All these actions are overridden to call methods in the bean:


Method reDrawJet is invoked, which calls simple utility method to invoke JS function which actually re-draws JET UI:


JET UI re-draw happens in JS function, which cleans Knockout.JS nodes and reapplies current JET model bindings:


JET -> ADF call is made through JET Composite event. This event is assigned with JS function implemented in ADF Faces context. This allows to call JS located in ADF Faces, without changing JET Composite code. I'm using regular ADF server listener to initiate JS -> server side call:


ADF server listener is attached to generic button in ADF Faces:


ADF server listener does it job and applies received key to set current row in ADF. Which automatically triggers ADF form to display correct data: