Watson analytics with nodejs integration

Get a key,from here

https://developer.ibm.com/api/view/id-160:title-Watson_Analytics

Go to the manage api section and get these two parameters.

X-IBM-Client-Id c1a4d636-51c1-4aad-
X-IBM-Client-Secret gC3pL3sG2eC0cK4kB3mK

Open postman and make a put request to

https://api.ibm.com/watsonanalytics/run/oauth2/v1/config

In body of put,enter like  this.

{
     "ownerName": "asdf",
     "clientName": "sdfasd",
     "redirectURIs": "https://localhost:6001",
     "ownerPhone": "9999999999",
     "ownerCompany": "asdf",
     "ownerEmail": asdfasdfs@sdfsd.com
}

replace junk with your info, this needs to be done once and put response will get the info you need.

Next time you can do a get on above url with headers, it will spit out the client info to you.

You are all set to push docs to analytics.

But we  have client id only and we need auth  code also, if you try without that you get below error.

"Failed to verify OAuth information. Transaction ID=42

so lets get the auth code

Open a browser and paste this.

https://api.ibm.com/watsonanalytics/run/clientauth/v1/auth?scope=userContext&response_type=code&client_id=-4cf7-89e2-94af7fd6180c&redirectui=http://realhost.org

Make sure you redirect url is same and is reachable, else you will get some background database not reachable error.

give permissions to the app.

You will get a call back with authcode likethis.

http://sdfsdfsdf:6001/?code=7qI2rTRi%2FrqOZkAvX61p835QFP7zkclb1NOyp3RI6NwN7wF%2Bm%2BaRUYFpN5ALvYK4RrN0netLwyFpBQmFjv%2Fc0t%2BC%2B4jOxEsnHW%2F5hPAJxBvI2YU3h3Nuk16iM0%2BW9AfP%2FF9AfK%2BrZgsHHEvMB9r1YAGq3m%2BqGR%2FtnfWrDptChBzhZyMv5LEnpm3p6O8cTI7yExGH%2FecXouP2KpwPB6y3ATDoIacoZUUcKZZlcSebqchDrx%2F5n2RWIxk5fSnWO%2BkQJwo%2BrLggmjLUXpMPeaA0qCOfX5P5gd3iU7spOgh5eA%3D%3D#/

Use this code in our node app, valid for 1 hour.

Don’ know why ibm confused so much with auths and tokens but friends.. one more step

we need access token before we get the data assets i know it sucks and sounds redundant.

use a decoder service to decode encoded code

Now we use this auth code to get token

https://api.ibm.com/watsonanalytics/run/oauth2/v1/token

Ref these docs, i could pass from here, tried different grant_type, all of them which were assigned to me in config , but none worked.

{
     "error": "invalid_client"
}

===================Not working….. from here…..so i moved to another analytics solution.. mailed ibm india team but no reply..

Ref.

github repo. followed instructions but didnt work .

https://github.com/WatsonAnalytics/oauth2-nodejs

https://developer.ibm.com/watson-analytics/#getstarted

https://community.watsonanalytics.com/discussions/questions/23305/automatic-upload-of-datasets-through-rest-api.html

Other errors encountered.

These were due to different tenants, i was using for  calling the auth requests or may be i was using localhost instead of actual reachalbe url of my app.(This one got fixed after using reachable url)

error_description": "Invalid grant type. TransactionI

couldn’t proceed after this and tried lot of things, but didn’t work.I tried with both Partner accounts and general public free accounts.

Embed dynamic JavaScript in angular page using controller injection

To insert a widget, lets take trading view widget if you want to insert javascript widget which needs to  be loaded on a page, you need to inject it inside the html page.

It won’t load if you are inserting the script directly in the html page.

You need to insert it via controller.

look at the controller code

app.controller('ChartsController',function($scope){

$scope.init = function(){

console.log("called on scope init");

$(' <div class="tradingview-widget-copyright"><a href="https://www.tradingview.com/symbols/NSE-ICICIBANK/" rel="noopener" target="_blank"><span class="blue-text">ICICIBANK</span> <span class="blue-text">chart</span> by TradingView</a></div> <script type="text/javascript" src="https://s3.tradingview.com/tv.js"></script> <script type="text/javascript"> new TradingView.widget( { "autosize": true, "symbol": "NSE:ICICIBANK", "interval": "D", "timezone": "Asia/Kolkata", "theme": "Light", "style": "8", "locale": "en", "toolbar_bg": "#f1f3f6", "enable_publishing": false, "withdateranges": true, "hide_side_toolbar": false, "allow_symbol_change": true, "watchlist": [ "NSE:RNAM", "NSE:LUPIN" ], "details": true, "hotlist": true, "calendar": true, "news": [ "stocktwits", "headlines" ], "studies": [ "IchimokuCloud@tv-basicstudies", "BB@tv-basicstudies", "RSI@tv-basicstudies", "Volume@tv-basicstudies" ], "container_id": "tradingview_7e6c5"} ); </script>').appendTo('#tradingview_7e6c5');

}

});

Add a route in the router section of the app.js

when('/charts', {

templateUrl: 'views/charts.html',

controller: 'ChartsController'

}).

Add this  to the chart.html

<div ng-init="init()" class="tradingview-widget-container">

<div id="tradingview_7e6c5"></div>

<div class="tradingview-widget-copyright"><a href="https://www.tradingview.com/symbols/NSE-ICICIBANK/" rel="noopener" target="_blank"><span class="blue-text">ICICIBANK</span> <span class="blue-text">chart</span> by TradingView</a></div>

</div>

You can embed javascript widgets from trading view like this.

nodejs gzip response on chunks are junk

If you are trying to call a request and trying to parse the response and getting

junk characters on the buffer concat of chunks, do this.

Change the header accept encoding.

"accept-encoding": "br" // remove gzip if present

You can check the encoding of response sent via server using this.

console.log('server encoded the data as: ' + (res.headers['content-encoding'] || 'identity'))

once you know what kind of encoding, remove it from the header and fire request again.

instead of pushing chunks to the array you can directly concat the string to the body.

body+=chunk.toString();

// Sample code

sample post code on server.js (node)

app.post('/app/call, function(req,res){

var http = require("https");

var response;

var options = {

"method": "POST",

"hostname": "xx",

"port": null,

"path": "/v3/topics",

"gzip":"true",

"headers": {

"origin": "https://xxi.com",

"accept-encoding": "br",

"accept-language": "en-US,en;q=0.8",

"authorization": "Bearer 7f850f07-xxxxxx",

"content-type": "application/json",

"accept": "*/*",

"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36",

"referer": https://sxxxx,

"authority": "xxxxx",

"cache-control": "no-cache",

}

};

var req = http.request(options, function (data) {

var chunks = [];

var body;

data.on("data", function (chunk) {

//console.log('server encoded the data as: ' + (res.headers['content-encoding'] || 'identity'))

chunks.push(chunk);

body+=chunk.toString();

});

data.on("end", function () {

// body = Buffer.concat(chunks);

console.log(body.toString());

response = body.toString();

res.write(response);

res.end();

});

});

req.end();

});

Technorati Tags: junked characters on gzip response via nodejs request module

Run nodejs applications on cloud with all dependencies available on cloud

https://npm.runkit.com

How to post data from angular to nodejs backend and parse response

Make the request in angular

app.js code , client side in our case.

$scope.getTor = function(){ // getTor is bound with ng-click in my html with my controller which has this method.

var torMcItems=$http({ // i want response in this variable.

method: 'GET',

url:'/api/tor' // my backend in server.js

})

.success(function(response){

response = JSON.parse(response);

torItemms=response.items;

}).error(function(err){

}

);

};

///////////////////////////////////////////////////////////

Now server side of the nodejs app , the below method is in server.js.

app.get('/api/tor', function(req, res){ //handle which was called in app.js

var request = require('request');

var options = {

url: 'https://theoldreader.com/reader/api/0/stream/contents?output=json&s=user/-/label/mcrecos',

headers: {

'Authorization':'GoogleLogin auth=emxxxxxxxC'

}

};

let body = [];

request(options).on('data', function(data)

{

//console.log(data.toString());

body.push(data);

}).on('end', function(){

body = Buffer.concat(body).toString();

res.setHeader('Content-Type', 'application/json');

res.write(JSON.stringify(body));

// want to convert the body to string , we will make it json object in the app.js using  JSON.parse

res.end();

});

} );

Unexpected token o in JSON at position

Try out on jslint and see if the json is valid, try to fire a JSON.parse(your string) in the chrome developer tool and see if you get the same error.

If you are using nodejs and this is a request from the server.js to fetch data from external system, you might be getting the data in chunks and you will have to use a buffer to accumulate the body.

Once the body is created you can set response headers and send the body back to the angular.

Use Json.stringify to send it back to client and use JSON.parse(incomingstring) to make the json object out of the string.

server.js =======Json.stringified(body) == going as string to client ====JSON.parse(incoming response)==> use the json object in app.

theoldreader , how to get the api working?

Get the token.

Use windows curl and fire.

curl –k -d "client=YourAppName&accountType=HOSTED_OR_GOOGLE&service=reader&Email=test@krasnoukhov.com&Passwd=..." https://theoldreader.com/accounts/ClientLogin

You will get the token save it for further calls.

To get list of items,fire the below

E:\tools\curl>curl -k  "https://theoldreader.com/reader/api/0/stream/items/ids?output=json&s=user/-/state/com.google/reading-list" -H "Authorization: GoogleLogin auth=emJtzTPu8xyYqxxxxx"

Now loop through the items contains with a for loop and operate on the content of the feeds.

Get all items list for a label(ET) in oldreader

curl -k  "https://theoldreader.com/reader/api/0/stream/items/ids?output=json&s=user/-/label/ET" -H "Authorization: GoogleLogin auth=emJtzTPu8xxx”

Get a specific item with id

curl -k  "https://theoldreader.com/reader/api/0/stream/items/contents?output=json&i=5ad626345f45b7b2c2013aa5" -H "Authorization: GoogleLogin auth=emJtxxxxxxxxxx”

Hope it helps.

How to install openshift origin on laptop locally in 5 minutes

Install vagrant

Run t

$ vagrant init openshift/origin-all-in-one

$ vagrant up --provider=virtualbox

///////////////////

default: You can now access the OpenShift console on: https://10.2.2.2:8443/console
   default:
   default: Configured users are (<username>/<password>):
   default: admin/admin
   default: user/user
   default: But, you can also use any username and password combination you would like to create
   default: a new user.
   default:
   default: You can find links to the client libraries here: https://www.openshift.org/vm
   default: If you have the oc client library on your host, you can also login from your host.
   default:
   default: To use OpenShift CLI, run:
   default: $ oc login https://10.2.2.2:8443

////////////////////////////////

Ssh in to machine

$ vagrant ssh

If you get any cygwin warning from windows, do this.

> set CYGWIN=nodosfilewarning

> vagrant ssh

You will get ssh to the machine.

[root@localhost vagrant]#yum install wget

[root@localhost vagrant]#wget http://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo -O /etc/yum.repos.d/epel-apache-maven.repo

[root@localhost vagrant]# yum install apache-maven

First time maven on a pom based project will take a long time to update.

kali apt-update slow internet connection

IF you are also struggling with following symptoms, please keep reading.

• Slow apt-get update on kali
• Slow apt-get upgrade on kali rolling repo
• The repository 'http://http.kali.org/kali kali-rolling Release' does not have a Release file.
• Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
• Slow internet connection on kali

you need to install the apt-get https package to enable faster speeds for apt-get

once installed edit /etc/apt/sources.list file to

deb https://http.kali.org/kali kali-rolling main contrib non-free

Try the apt-get update after this speed should go in mega bytes.

Also I found synaptic package manager to fix all my repository issues.

#apt-get install synaptic // will install the package manager, also has provision for fixing broken packages, it fixed a lot of issues my repos.

If you are facing issues with release file not available or repository is not secured and its dangerous to install from unsecured registries. you can simply add the key using the below command.

wget -q -O - archive.kali.org/archive-key.asc | apt-key add

not able to run apt --fix-broken install in kali

if you have multiple versions of a  package and there is a conflict of dependencies on your system.

It might happen that you wont be able to install anything or do an apt-get autoremove or apt-get –f install  or apt-get install any thing.

You can copy the package status file

$cp /var/lib/pkg/status /backup loc

Now $ leafpad /var/lib/dpkg/status

Edit the controversial package, make sure you maintain the syntax else you will get parse error

while running apt-get again from cmdline.

Remove the dependencies for a while, we can do an apt-get update/install later on.

Once you have removed the problematic entry, for example the package or its dependency which was creating the problem.

Run the apt --fix-broken install –f again and apt-get update after this.

Install a package without apt-get on kali

Get the zip/tarball from wget

wget tar/bz2
bzip2 file
tar xzf // will give tar
tar xvf // will give folder
./configure
make
make check
make install

Build whatsapp plugin on wireshark

The old code on GitHub for building whatsapp.so isnt working anymore with the new libwireshark-dev.

You need to clone the git on your kali system using

$git clone https://github.com/davidgfnet/wireshark-whatsapp

mkdir build

cd build

cmake ..

but this will error out because you need to update the CMakefile.txt with the wireshark include dirs and cmake install directories(if you dont have cmake, do an apt-get install cmake and make sure you include the dir in the cmake file)

Sample entries

set(WIRESHARK_INCLUDE_DIRS /usr/include/wireshark)
set(GCRYPT_INCLUDE_DIR /usr/local/include)

Just include them in the CMakeLists.txt top section, make sure Gcrypt is installed)

Good idea to read the readme again on the above git, you need some kind of dependencies before this build can succeed.

There were some modifications required to make this git build work with the newest version of libwireshark-dev, I created a pull request on the git and patched the code.

If the owner accepts my pull you can use the git mentioned above, you can also use my forked one here

$ make install

  P.S Make sure you do apt-get update before doing any thing and include dependencies before running cmake or make.

// Wireshark version 2.4.5 libwireshark-dev 2.4.5

Minishift ( the openshift origin local installation) not starting on windows

To start minishift on windows, you need to have two prereq.

ssh client should be there in path, either cygwin or ssh runtime which comes with windows 10, you can enable it from the features.

also you will need to set the cygwin waring flat either in the command prompt from where you are starting minishift or from the profile. cmd > set CYGWIN=nodosfilewarning // now start mini

cmd> minishift start –vm-driver=virtualbox

This should start the cluster and , minishift status should give you that openshift is running.

You can also try minishift update , it might fix errors you getting.

Also you can set the size of the instances if you are getting space issues while starting minishift, this can be set as a command line parameter while starting minishift. //--disk-size 10g

Troubleshoot:

If you want to recreate the machine, you can delete it from the ~/.minishift/machines/minishift folder and its json and restart minishift, it will pull the openshift 3 again with layers and will start afresh, this is just in case your current config has gone bonkers.