Search This Blog

Monday, November 18, 2019

Move local rocketchat to production

If you have a dev environment on VSC and you want to move to production by creating docker images
please read ahead

create docker out of your meteor installation.
meteor build --server-only --directory /tmp/rc-build cp .docker/Dockerfile /tmp/rc-build cd /tmp/rc-build docker build -t someimage .

check your local meteor mongo port and take a dump.


mongodump -h 127.0.0.1 --port 3001 -d meteor  --forceTableScan  


// dbname is meteor for dev by default use the local mongodumb cli in ubuntu if not install mongo-tools.
//you will have to move this folder as a gzip file to the mongo container and do a restore.
tar -zcvf meteor.tar.gz meteor/   

docker cp meteor.tar.gz mongo:/ ----------- mongo is container name and will place gz file in the root.



mongorestore -d rocketchat dump/meteor // db name if rocketchat by default for offical images

//make sure to check db names while importing use a tool like robot3 to have mongo gui



to run the new images run below.
$ docker run --name mongo -d mongo:4.0 --smallfiles --replSet rs0 --oplogSize 128
$ docker exec -ti mongo mongo --eval "printjson(rs.initiate())"

// default rocket.chat images look for mongo container to connect to on 27017 and with name mongo.

 docker run --name rocketchat -p 80:3000 --link mongo --env ROOT_URL=http://localhost --env MONGO_OPLOG_URL=mongodb://mongo:27017/local -d someimage

//docker file for rocket.chat
https://github.com/RocketChat/Rocket.Chat/blob/develop/.docker/Dockerfile

Wednesday, October 23, 2019

upload xl/CSV to S3 and process it via lambda for db insert

create a bucket 
create a lambda function and add trigger function as S3
when you create lambda a role will be created for you, add additional policy to it , dynamodb and S3 policy
Now add the following code in the lambda_handler function of the lambda
========================================
import json
import boto3

s3 = boto3.client('s3')
dynamodb = boto3.resource('dynamodb')

def lambda_handler(event, context):
    # TODO implement
   
    bucket = event['Records'][0]['s3']['bucket']['name']
    key = event['Records'][0]['s3']['object']['key']

    obj = s3.get_object(Bucket=bucket, Key=key)

    rows = obj['Body'].read().decode('utf-8') .split('\n')

    table = dynamodb.Table('entity')
   
    print(len(rows))
    with table.batch_writer() as batch:
        for row in rows[:-1]:
            batch.put_item(Item={

                'name':row.split(',')[0],
                'address':row.split(',')[1]
            })
           
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }

===================================
^^ make sure you have a dynamodb with table name entity and field name,address 
you can add as many fields as you want.

Now upload the csv to s3 and you should see lambda processing your xl and inserting to dynamo.

=============================
Sample csv
name,address,city
target,23230,austin
walmart,77707,houston
macy,80808,dallas


First row will be skipped as per the lambda for loop 

Friday, October 18, 2019

Connect to posgres on aws problem

create a new rds instance but make sure you select advanced options
and create a starter database else you won't be able to connect to
posgres and get errors.

so an initial db has to be there if you connecting via sqlworkbench
or eclipse or psql.

connect to ec2 via windows

Download the pem file from aws and convert to ppk via the puttygen,
Create a new session and in auth tab browse to ppk
enter user name as ec2-user and url and public dns or ec2