Installing multiple versions of node and npm using nvm

 Environment: Ubuntu / Debian

$curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash

add following on your .bash_profile or .bashrc if not added automatically when running above command 

export NVM_DIR="$HOME/.nvm"

[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  

[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"  

Logout and login terminal 

Now if you want to install node version 20.10.0 then issue 

$nvm install 20.10.0

 

Now if you want to use some other version (eg. node version 21.0.0) and want to use it

$  nvm ls-remote

$ nvm  install 21.0.0

$ nvm use  21.0.0

Containerizing Node.js, Django, Postgres, Celery, RabbitMQ and Elasticsearch

 

 

Environment: Ubuntu 23.10
Docker Version:   26.1.2
Docker Compose Ver: 2.27

1. Dockerfile for Nextjs front end placed parallel to package.json file

FROM node:20.12.2-alpine
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 3000
CMD ["npm", "run", "dev"]

1.a. .env

NEXT_PUBLIC_BASE_URL=http://192.168.x.x:8000

 

2. Dockerfile for Python backend placed above src folder 

FROM python:3.11.4-alpine3.18
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
WORKDIR /usr/src/app
COPY ./src/requirements/requirements_dev.txt /usr/src/app
COPY ./src/requirements/base.txt /usr/src/app
RUN pip install --upgrade pip && \
    apk update && \
    apk add postgresql-dev gcc python3-dev musl-dev jpeg-dev zlib-dev curl && \
    pip install -r requirements_dev.txt
COPY ./src/.env /usr/src/app
RUN mkdir /usr/src/app/logs
COPY ./entrypoint.sh /usr/src/app
RUN sed -i 's/\r$//g' ./entrypoint.sh
RUN chmod 777 ./entrypoint.sh
COPY ./src /usr/src/app/
RUN chmod 777 ./es_index.py
ENTRYPOINT ["./entrypoint.sh"]

2.a. entrypoint.sh

#!/bin/sh

export POSTGRES_HOST="db"

if [ $DATABASE = "postgres" ]
then
    echo "Waiting for postgres to load..."

    while ! nc -z $POSTGRES_HOST $POSTGRES_PORT; do
        sleep 0.1
    done

    echo "postgres ready to run"
    python manage.py makemigrations
    python manage.py migrate --no-input
fi

python manage.py createsuperuser --noinput --username $DJANGO_SUPERUSER_USERNAME --email $DJANGO_SUPERUSER_EMAIL
python es_index.py

exec "$@"

2.b. vi .env

# secret key
SECRET_KEY=ac6cd19772005f224cefb220060037bf15f35a6e3e1826a1095a26fc

# database stuffs
DATABASE=postgres

DB_ENGINE=django.db.backends.postgresql
POSTGRES_DB=courses
POSTGRES_USER=postgres
POSTGRES_PASSWORD=xxxxx
POSTGRES_PORT=5432
POSTGRES_HOST=db

# superuser setup
DJANGO_SUPERUSER_USERNAME=admin
DJANGO_SUPERUSER_PASSWORD=admin
DJANGO_SUPERUSER_EMAIL=admin@gmail.com

# smtp setup
EMAIL_HOST_USER=courses@xxxxxx.org
EMAIL_HOST_PASSWORD=titygjzkngqtzoek

# celery stuffs
#CELERY_BROKER_URL=amqp://guest:guest@localhost:15672/
#CELERY_RESULT_BACKEND=db+postgresql://postgres:admin@localhost:5432/celery_tasks

# Adjust for dockerizatin
CELERY_BROKER_URL=amqp://myuser:mypassword@rabbitmq:5672
CELERY_RESULT_BACKEND=db+postgresql://postgres:admin@db:5432/celery_tasks

ADMIN_USER=xxxxxx@xxxxxx.org

# elastic
#ES_HOST=localhost
ES_HOST=elasticsearch
ES_NUMBER_OF_SHARDS=1
ES_NUMBER_OF_REPLICAS=0
ES_USE_SSL=False
ES_PORT=9200
ES_INDEX=seepalaya

3. Dockerfile for Celery 

FROM python:3.11.4-alpine3.18
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
WORKDIR /usr/src/app
COPY ./src/requirements/requirements_dev.txt /usr/src/app
COPY ./src/requirements/base.txt /usr/src/app
RUN pip install --upgrade pip && \

    apk update && \
    apk add postgresql-dev gcc python3-dev musl-dev jpeg-dev zlib-dev && \
    pip install -r requirements_dev.txt
COPY ./src/.env /usr/src/app
RUN mkdir /usr/src/app/logs
COPY ./src /usr/src/app/

3.a .env

similar to above 

 

4. Dockerfile for Postgres

FROM postgres:15.7-alpine3.18
ENV PG_MAX_WAL_SENDERS 8
ENV PG_WAL_KEEP_SEGMENTS 8

#Set timezone
ENV TZ=Asia/Kathmandu
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

# Copy the custom initialization script to the Docker image
COPY create_celery.sh /docker-entrypoint-initdb.d/
COPY *.sql /tmp/
COPY populate_ep.sh /docker-entrypoint-initdb.d/

# Grant execution permissions to the script
RUN chmod +x /docker-entrypoint-initdb.d/create_celery.sh
RUN chmod +x /docker-entrypoint-initdb.d/populate_ep.sh
 

4a. create_celery.sh

#!/bin/bash
set -e

# Create additional databases
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" <<-EOSQL
    CREATE DATABASE celery_tasks;
EOSQL

4b. populate_ep.sh

#!/bin/bash
set -e

# Function to check if the database exists
db_exists() {
    psql -U "$POSTGRES_USER" -tc "SELECT 1 FROM pg_database WHERE datname = 'pustakalaya'" | grep -q 1
}

# Create the database if it does not exist
if ! db_exists; then
    psql -U "$POSTGRES_USER" -c "CREATE DATABASE pustakalaya"
fi

# Create the user and grant privileges
psql -U "$POSTGRES_USER" <<-EOSQL
    DO \$\$
    BEGIN
        IF NOT EXISTS (SELECT FROM pg_catalog.pg_user WHERE usename = 'pustakalaya_user') THEN
            CREATE USER pustakalaya_user WITH PASSWORD 'pustakalaya123';
        END IF;
    END
    \$\$;
    GRANT ALL PRIVILEGES ON DATABASE pustakalaya TO pustakalaya_user;
EOSQL

psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname=pustakalaya < /tmp/pustakalaya.sql

4.c. .env

POSTGRES_DB=courses
POSTGRES_USER=postgres
POSTGRES_PASSWORD=xxxxx                

 

5. docker-compose.yml

version: "3.8"
services:
  db:
    build:
      context: seepalaya-new-database
      dockerfile: Dockerfile.postgres
    container_name: seepalaya-db
    env_file:
      - ./seepalaya-new-database/.env
    volumes:
      - ./db/:/var/lib/postgresql/
    networks:
      - seepalaya-new
    restart: always


  seepalaya-backend:
    build:
      context: seepalaya-new-backend
      dockerfile: Dockerfile
    container_name: seepalaya-backend
    command: "python manage.py runserver 0.0.0.0:8000"
    ports:
      - 8000:8000
    env_file:
      - ./seepalaya-new-backend/src/.env
    environment:
      - CELERY_BROKER_URL=amqp://myuser:mypassword@rabbitmq:5672
    depends_on:
      - db
    networks:
      - seepalaya-new
    restart: always

  rabbitmq:
    image: rabbitmq:3-management
    ports:
      - "5672:5672"
    environment:
      - RABBITMQ_DEFAULT_USER=myuser
      - RABBITMQ_DEFAULT_PASS=mypassword
    networks:
      - seepalaya-new
    restart: always

 seepalaya-celery:
    build:
      context: seepalaya-new-celery
      dockerfile: Dockerfile
    container_name: seepalaya-celery
    command: celery -A config worker -l info
    environment:
      - CELERY_BROKER_URL=amqp://myuser:mypassword@rabbitmq:5672
    depends_on:
      - rabbitmq
      - seepalaya-backend
    networks:
      - seepalaya-new
    restart: always

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.12.1
    environment:
      - discovery.type=single-node
    ports:
      - "9200:9200"
    networks:
      - seepalaya-new
    restart: always
 
  seepalaya-front:
    build:
      context: seepalaya_new
      dockerfile: Dockerfile     
    container_name: seepalaya-front
    env_file:
      - ./seepalaya_new/.env
    ports:
      - "3000:3000"
    networks:
      - seepalaya-new
    restart: always

networks:
  seepalaya-new:
    driver: bridge

 

Installing multiple versions of python on linux using pyenv and create virtual environment.

Environment: Debian12 

Default python version on debian12 is 3.11 but I need 3.10 to run my flask based back end.

1. Check default python version

python3 --version

 

 2. Install version 3.10.13 (chose the version you need)

cd
sudo apt install curl git gcc make libssl-dev libbz2-dev
sudo apt install  python3-setuptools
sudo apt install python3-pip
curl https://pyenv.run | bash
cd .pyenv/bin
./pyenv install --list
./pyenv install 3.10.13
cd ~/.pyenv/versions/3.10.13/bin/
./python3.10 --version

 

3.  Create Virtual Environment

cd ~/.pyenv/versions/3.10.13/bin/
./python3.10 -m venv ~/.venv310

4. To activate virtual environment

cd
source .venv310/bin/activate
(.venv310):~$

 

 

 

Top 4 steps to secure debian / ubuntu

1. Security Update

grep security /etc/apt/sources.list > /tmp/security.list
sudo apt-get upgrade -oDir::Etc::Sourcelist=/tmp/security.list'
 

2. IPS/IDS using fail2ban

apt-get install rsyslog
apt-get install fail2ban
 

3. Install and enable firewall using ufw

vi /etc/myufw.sh

#!/bin/bash
ufw=/usr/sbin/ufw
$ufw disable
$ufw default deny incoming
$ufw default allow outgoing
$ufw allow ssh
$ufw allow https 
$ufw allow 5000 
$ufw allow 3000 
$ufw allow 7000
$ufw - force enable

$sudo chmod +x /etc/myufw.sh

$ sudo chmod /etc/myufw.sh

$ sudo iptables -L -n 
 

4. Enable only key based authentication:

vi /etc/ssh/sshd_config and check following 2 lines
PasswordAuthentication no
PermitRootLogin no

$sudo systemctl restart ssh

 

Convert .flv (Google Videos) to .mpeg using ffmpeg

Install ffmpeg package

get install ffmpeg

The basic command is:

mpeg -i youtube.flv youtube.mpg

To convert all the files in one shot:

for i in *.flv;
do
ffmpeg -i $i `basename $i .flv`.mpg;
done

Dyndns over NCELL Connect

Setting up NCELL Mobile Broadband & Dynamic DNS on OLPC School Server (NEXS)

(Senario: Fedora 9-0.6 , OLPC School Server, Network Manager can not be run for there is no GUI, so dial using wvdial on Terminal)

Steps:
1. Installed following 2 packages (Had to compile from source)

# rpm -ivh usb_modeswitch-data-20101222-1.fc9.noarch
# rpm -ivh usb_modeswitch-1.1.6-1.fc9.i386


2. Check to see if the Modem(Datacard) is being detected :Huawei E160/E220 USB Stick

# wvdialconf

Found a modem on /dev/ttyUSB0

3. It may be necessary write a following udev rule if the modem doesn't get switched to modem mode automatically

# cat > /etc/udev/rules.d/99-ncell-connect.rules <# Ncell Connect switch mode to usbserial (E1550 device id => 12d1:1446)
SUBSYSTEM=="usb", ATTRS{idProduct}=="1446", ATTRS{idVendor}=="12d1", RUN+="/lib/udev/modem-modeswitch --vendor 0x12d1 --product 0x1446 --type option-zerocd"
EOT


4. Write a wvdial dailer file

# vi /etc/wvdial.conf
[Dialer Defaults]
Init1 = ATZ
Init2 = ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
Stupid Mode = 1
Modem Type = Analog Modem
ISDN = 0
Phone = *99#
Modem = /dev/ttyUSB0
Username = web
Dial Command = ATDT
Password = web
Baud = 460800


5. Run the dialer

# wvdial

Now, Setting up Dyndns

1. Get a free dyndns domainname, its free, signup and on dyndns.com. you can get upto 5 free domainnames.

2. Install dyndns.org client package.

# yum install ipcheck
( The above didn't work for me, I went to my debian machine and did following to get rpm out of ipcheck deb package)

# apt-get install ipcheck
# cd /var/cache/apt/archieve
# alien -r ipcheck_0.233-1_all.deb

Then,

copied ipcheck-0.233-2.noarch.rpm over to server and installed using

#rpm -ivh ipcheck-0.233-2.noarch.rpm


3. Get the domainname, username and password ready and do following

# python /usr/sbin/ipcheck.py --makedat -i ppp0 username password hostname

(Eg. python /usr/sbin/ipcheck.py --makedat -i ppp0 ole ole lal.homelinux.org)


4. Wait for a minute and ping lal.homelinux.org, it should resolve to the IP address of your ppp0 interface.

NCELL Connect in Linux, Ubuntu

sudo su
# apt-get install usb-modeswitch
# cat > /etc/udev/rules.d/99-ncell-connect.rules <# Ncell Connect switch mode to usbserial (E1550 device id => 12d1:1446)
SUBSYSTEM=="usb", ATTRS{idProduct}=="1446", ATTRS{idVendor}=="12d1", RUN+="/lib/udev/modem-modeswitch --vendor 0x12d1 --product 0x1446 --type option-zerocd"
EOT

Linux System Cloning using "dd"

First check the partition table first using following command
fdisk -l

Device Boot Start End Blocks Id System
/dev/sda1 1 9 72261 83 HPFS/NTFS
/dev/sda2 10 75 530145 82 Linux swap
/dev/sda3 76 467 3148740 fd Linux raid autodetect
/dev/sda4 468 2200 13920322+ 83 Linux

Simple but may take a lot of time ( over 10 hours is no surprise)
dd if=/dev/sda of=/dev/sdb

If you think there is no point in cloning swap areas and empty partition, clone each partition

Note: Both drives are partitioned exactly same. If you have different brand harddrives, make sure each partition on second drive must be equal to or greater than first drive partitions. Also make sure File system ID should match for second drive also.

dd if=/dev/sda of=/dev/sdb bs=446 count=1
dd if=/dev/sda1 of=/dev/sdb1 ==> Clone NTFS partition
dd if=/dev/sda3 of=/dev/sdb3 ==> Clone RAID-1 partition

Space Matters

QuickBird
http://en.wikipedia.org/wiki/QuickBird

Vandenberg Air Force Base
http://en.wikipedia.org/wiki/Vandenberg_Air_Force_Base

Space Shuttle lands in California
http://news.bbc.co.uk/2/hi/science/nature/8062773.stm

European Austronauts to be named
http://news.bbc.co.uk/2/hi/science/nature/8057633.stm

Space station module handed over
http://news.bbc.co.uk/2/hi/science/nature/8053963.stm

Hubble's New Spectrograph Fitted
http://news.bbc.co.uk/2/hi/science/nature/8054055.stm

Lift-offs for European Telescopes
http://news.bbc.co.uk/2/hi/science/nature/8050157.stm

Edwards Air Force Base
http://en.wikipedia.org/wiki/Edwards_Air_Force_Base

Cape Canaveral Air Force Station (CCAFS)
http://en.wikipedia.org/wiki/Cape_Canaveral_Air_Force_Station

Kennedy Space Center (KSC)
http://en.wikipedia.org/wiki/Kennedy_Space_Center

Shuttle blasts off to fix Hubble
http://news.bbc.co.uk/2/hi/science/nature/8044041.stm

Hubble fix mission set for launch
http://news.bbc.co.uk/2/hi/science/nature/8038257.stm

World's most daunting parking job
http://news.bbc.co.uk/2/hi/science/nature/8038257.stm

Space shuttle touches down safely
http://news.bbc.co.uk/2/hi/science/nature/8177528.stm

Linux Training of Trainers in Manila

About myself:
I did my intermediate in Science, bachelors in Management and masters in Information Technology. I first used computer in 1994 when my uncle who ran a long distance call center in Kathmandu bought computer for the sake of providing Email/Internet service. I learnt computer faster in comparison to other colleagues. I didn't know at that time that this tool will become the way of my life.

I joined some computer training institute and did well. Joining an ISP, at being invited by an old friend who we had studied together during our intermediate and who had then become an experienced system administrator, was a turning point in my life. I worked and learned. Now, I am working in a Linux Localization Project and i'm one of the lead developers there. We have recently released the Nepali Version of Debian GNU/Linux named 'NEPALINUX'.

Training of Trainers:
Linux Training of Trainers
UP, Philippines
4th - 16th Dec. 2006

Jointly hosted by :
International Open Source Network, IOSN
InWEnt-Capacity Building International,Germany
University of the Philippines.

A colleague at Madan Puraskar Pustakala, where we work on developing Nepalinux, informed about the training and he further asked if I would be interested to attend. Despite my soon-appearing exam at the college, I agreed. Mr. Yusof, Dr. Alvin and Mr. Eric Pareja who usually hang around #iosn irc co-ordinated to send me an invitation letter and make arrangements for releasing DSA through UNDP-Kathmandu.

The following 2 days I was busy with air-ticket collection, visa application and other arrangements.

My First Overseas Trip Alone:
Kathmandu-Bangkok-Manila. I was afraid cause this was my first travel alone ever. I had a 16 hrs long transit at Bankok airport. Fortunately, I met a Nepali novice traveller just like me and we became friends. Fortunately again, he was also going to Manila, the same place I was going. Half of my tension and nervousness was gone when I knew about this. We landed at the manila airport at around 11 am. This new friend was lucky that he had somebody waiting to pick him up. He walked away almost instantly when he saw his receiver and didn't look back at me even once as he walked away. I was on my own. I bought a telephone Sim card (telco) costing 3$ and made call to the event organizer/manager Dr. Alvin B. Marcelo. He suggested me to get a airport taxi. To my surprise the taxi driver at his mid 50s spoke very good English. Soon the taxi took us inside the University Area. I had never seen such a huge university area. I had to get off the taxi few times to make inquiry before we could finally locate the University Hotel. A room was supposed to be booked for me in the hotel.

The University Hotel:
The room was nice and had three sub rooms. Two sub-room were to be taken by 2 Mongolian colleagues who arrived only 2 days after my arrival. Later I found out that the receptionist had problem identifying the Mongolians' name as well and they had been given a different room the earlier night.

First day of the Training:
Dr. Alvin Marcelo, the manager of the event, whom I had only talked over the telephone from the airport was very kind to pick me up at the Hotel the next day. He drove me to the UP Diliman Interactive Learning Center - the Computer lab -just a few hundred meters away. He further suggested me on taking a short-cut to the lab the following days. As I entered the lab, i realized that I was late. A lady instructor who was already giving linux installation instruction in the lab asked me to introduce myself to the class before having seat. I took the front seat and shook hands with a healthy looking guy who was already seated there besides me. He introduced himself as Mr. Ron. He was to become my buddy for coming few days, until one day he stopped showing up in the lab unnoticed. He was knowledgeable but was not very comfortable with linux system. I helped him with few commands and he quickly realized that I was fairly good at linux..

The lab is something:
The lab was indeed the best lab I had ever seen. I was surprised to see that all 24 "Legend" Systems had been donated by the embassy of China in Philippines. Flat monitor, fresh 40 gb hdd, colorful set of speaker, mouse, keyboard and CPU placed gracefully over the fresh looking computer table within that air conditioned lab all gave a very pleasant feeling. All the system were powered through a build-in UPS which gave beep signal whenever the power cable was unwittingly unplugged by participants. But when I noticed a projector hunging down from the lab-ceiling , I said to myself yes, this lab is something.

Snacks & Lunch Breaks:
Snacks were served twice a day in the room above the lab. The following two days, during the snack breaks I personally introduced myself to every single participants. Everyone greeted me with a smile as I was the only foreign student until the Mongolians came.

During the class instructor Myra had told about the places to find lunch around there and it would cost around 60 peso for lunch. Finding lunch was a challenge for me cause I didn't ate beef and beef being very popular, seemed to be there in all the lunch items that were displayed in the lunch restaurants. I found myself having roasted chicken and rice most of the time.

Lots of Instructors:
In the lab, all the participants were handed the objective materials and a set of ubuntu server and desktop CDs. Linux installation and HDD partitioning was very well covered by Instructor Myra. We got to see the lead instructor only on the 2nd day. He introduced himself as Mr. Eric Pareja. He had all the features that a very good instructor needed. A very good knowledge of the system, years of hands on experience, precise, clear and slow speech were the few weapons Mr. Eric Pareja was armed with. I personally have learned a lot from him not just about the computer and computer system but also about the teaching techniques.

Instructor Jeff, Ariel were equally good and knowledgeable.

LPI Exam:
When one day in the class we were informed that we are to take the LPI exam at the end of the training, I became little more serious and started following the objective seriously. As I went through the book, i realized that the book content where very carefully compiled and it contained many useful topics.

Thank god I passed the exam.

The Linux gurus:
The training was more of a fast review of everything. It was Training of Trainers after all!. I noticed some participants were really very good and well experienced. I realized how training out side one's own country is important. One starts viewing his work in a different way and starts seeing that there is scope for best of anything... I had a little chat with the philippino linux gurus who made the Bayanihan GNU/Linux. They had come there to give their small speech on Baynihan linux. I introduced myself as one of the NepaLinux GNU/Linux Developer. I went ahead and showed them Nepalinux in my laptop and they appreciated our work.

Trainees:
"At snack breaks and during lunch time, show our international participants your respect and hospitality" these are the words spoken by Mr. Eric addressing fellow philippino trainees. There were altogether 24 participants. I was form Nepal and besides myself, the other international participants were the 2 Mongolians who I shared the big hotel room with. Among the philippino participants some came from far away province.

I found most philippino trainees humble. Most of the participants greeted me with cute smile whenever our eyes met. A fellow trainee Mr. Benny became very close to us -me and the Mongolians. He was the only trainee who had car. He worked as a sysadmin for some other department within the university area. He took us to the city center, the big mall, bay side and the mega mall. He even helped me shift the hotel. Later after training he was to take us to the Subik area and the sea beach and were to have great time together scuba-diving and holding starfish. I am very grateful to Benny for taking us to see sea and other places and so are the mangolians (Dulmandakh and Nunhe) I am sure.

Miss Melissa was the only lady participant. She was cute and fashionable. She was the first lady I had seen who was so good with linux and it fascinated me a lot. She had started working for UP just a few years back after finishing her bachelors. We had a few very nice conversation during snacks breaks. We were to become very good email friends in the coming days.

FOSS Essentials Training 2007
March 17 - 18, we are organizing "FOSS Essentials Training 2007" right within our office premises Madan Puraskar Pustakalaya, Patan, Nepal.
http://www.fossnepal.org/?q=node/75

Organized by:
FOSS Nepal Community

Supporting Partners:
Center of the International Cooperation for Computerization (CICC), Singapore
National Information Technology Centre (NITC)
Madan Puraskar Pustakalaya (MPP)

The Training will focus on general Free And Open Source Software (FOSS) usage, migration from proprietary software to FOSS, licensing and intellectual properties rights, FOSS development management and various other FOSS issues.

This training will consist of lectures from experts, live practical sessions in laboratories, group discussions and projects. Our approach will be an informal training rather than a formal one. The trainees will get chance to interact with other participants and the trainers to a great extent from our informal lab sessions and group discussions.