EC2-spotter (merge spot and ondemand)

https://github.com/atramos/ec2-spotter/blob/master/classic/ec2spotter-launch

Mongo and Python - getting started

1. install pymongo {pip install pymongo}
2. from pymongo import MongoClient

conn=MongoClient()
db=conn.<dbname>
collection=db.<collection_name>

How to create unique index with pymongo

collection.create_index([('key',pymongo.ASCENDING)],unique=True)

Update command in Python

collection.update({'key':keyval},{'key':keyval,'field1':f1val},upsert=True)

Quick MongoDB commands

Create mongo database:
 use <database_name>

Create Collections
db.createCollection(name, options) 

db.collection.createIndex({"key":1},{unique:true})

db.collection.deleteMany({}) to delete all items

db.collection.drop() delete collection

Insert if not found otherwise update

db.people.update(
   { name: "Andy" },
   {
      name: "Andy",
      rating: 1,
      score: 1
   },
   { upsert: true }
)

regex testing tool

One of best site to test regex online

https://regex101.com/r/nG1gU7/27

Indic NLP Python resources list

1. https://github.com/anoopkunchukuttan/indic_nlp_library
2.https://github.com/goru001/inltk

CONCAT_WS magic concat in mysql

16

If you want to skip NULL values (but not empty strings), you can use CONCAT_WS() function:

CONCAT_WS( ', ',            -- Separator
           CONCAT_WS(' ', tadd.street_number, tadd.street_name),
           tadd.apt_number,  tadd.city, 
           tadd.postal_code, tadd.country
         ) AS Address

How to do fast query without inner join

refer
https://medium.com/squad-engineering/blazingly-fast-querying-on-huge-tables-by-avoiding-joins-5be0fca2f523

WITH user_ids AS
  (SELECT id
   FROM user
   WHERE account_id IN
       (SELECT generate_series(1,1000)))
SELECT purchase.id
FROM purchase
WHERE user_id IN
    (SELECT id
     FROM user_ids);

JOIN:

SELECT "purchase"."id"
FROM "purchase"
INNER JOIN "user" ON ("purchase"."user_id" = "user"."id")
WHERE "user"."account_id" IN
    (SELECT generate_series(1,1000));

Translation - Key terminologies

1. Edit Distance => Minimum character change required to match two strings. Also known as Levenshtein distance.

https://pypi.org/project/editdistance/0.3.1/ is python library to use for finding edit distance between two sentences.

Great tool to creare Embed Charts

use https://app.datawrapper.de/signin

How to read Mongo JSON in Python

Below is sample code to read Mongo JSON in python code.

Note: Every line have one Object in Mongo JSON

import json
file = open('../data/j.json','r')
jdata=[]
for line in file:
    jdata.append(json.loads(line))
totalSM=0for reportD in jdata:
    print reportD['key1']
    if 'key2' in reportD.keys():
        imD=reportD['key3']
        if '7' in imD:
            totalSM=totalSM+1            print totalSM

Install PHP composer

1,sudo curl -s https://getcomposer.org/installer | php
2.sudo mv composer.phar /usr/local/bin/composer