Remove unused Linux Kernels

Removing unused Ubuntu Linux Kernels.

When we install updates on Ubuntu Linux, the system gives a message free up kernel space message. Usually as updates are regularly downloaded & installed, the old versions exist, with their new counterparts. The old updates can be purged by issuing the following command.

The command cleans up all unused kernels on Ubuntu Linux.

$ > dpkg -l ‘linux-*’ | sed ‘/^ii/!d;/’$(uname -r | sed “s/\(.*\)-\([^0-9]\+\)/\1/”)‘/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d’ | xargs sudo apt-get -y purge

Upgrade Ubuntu from 16.04 to 18

If you are on Ubuntu 16.04 and need to move to 18, the following steps would help you achieve that. The upgrade is large and you would be inquired with a few questions while the up-gradation is happening.

First update the current distribution, with the following command

$ sudo apt update

$ sudo apt upgrade

This makes sure your Ubuntu is up to date. Next, upgrade the distribution of ubuntu

$ sudo apt dist-upgrade

This handles changing software dependencies with new versions of packages.

I then follow this up with:

$ sudo apt-get autoremove

This removes dependencies from uninstalled applications. You can do the same thing from the GUI desktop with the utility application Bleachbit. After that, it’s time to get things ready for the big upgrade with:

$ sudo apt install update-manager-core

Finally:

$ sudo do-release-upgrade

This will start upgrading your system to 18.04. Along the way, Ubuntu will ask you several questions about how to handle the upgrade.

Setting up MYSQL in CENTOS 6.9 Command Line

Prerequisite:

Whilst using MYSQL SERVER 5.7 or above kindly ensure to comment the line

#thread_concurrency = 8

 in the file (“/etc/my.cnf “  or “/etc/mysqld/my.cnf “ ) as its deprecated from version 5.7.

Installation Commands:

yum remove mysql57-community-release.noarch
yum clean all
yum install https://dev.mysql.com/get/mysql57-community-release-el6-11.noarch.rpm
yum install mysql-community-server

Now Start the Server using the following commands:

/etc/init.d/mysql start ## use restart after update

## OR ##

service mysql start ## use restart after update

chkconfig –levels 235 mysqld on

service mysql status

Note: IF service doesn’t start check logs for error using command ‘cat /var/logs/mysqld.log’, if table is damaged kindly run commands to fix

            /etc/init.d/mysqld start –skip-grant-tables && mysql_upgrade

            ## OR ##

            service mysqld start –skip-grant-tables && mysql_upgrade

        /etc/init.d/mysqld status

          ## OR ##

         service mysql status

If the UserName & Password for the server already exists then use command,

            mysql -u root -p

and enter the password to login.

If it’s a first time installation then use the following command to generate “temporary password” to login,

            sudo grep “temporary password” /var/log/mysqld.log

Example Output:

[Note] A temporary password is generated for root@localhost: -et)QoL4MLid

 

Next Follow the steps below to set up the server database (Optional):

 MySQL Secure Installation

  • Change root password
  • Remove anonymous users
  • Disallow root login remotely
  • Remove test database and access to it
  • Reload privilege tables

Start MySQL Secure Installation with following command

/usr/bin/mysql_secure_installation

Output:

Securing the MySQL server deployment.

Enter password for user root:

The existing password for the user account root has expired. Please set a new password.

New password:

Re-enter new password:

VALIDATE PASSWORD PLUGIN can be used to test passwords

and improve security. It checks the strength of password

and allows the users to set only those passwords which are

secure enough. Would you like to setup VALIDATE PASSWORD plugin?

Press y|Y for Yes, any other key for No: y

There are three levels of password validation policy:

LOW    Length >= 8

MEDIUM Length >= 8, numeric, mixed case, and special characters

STRONG Length >= 8, numeric, mixed case, special characters and dictionary                  file

Please enter 0 = LOW, 1 = MEDIUM and 2 = STRONG: 0

Using existing password for root.

Estimated strength of the password: 100

Change the password for root ? ((Press y|Y for Yes, any other key for No) : y

New password:

Re-enter new password:

Estimated strength of the password: 50

Do you wish to continue with the password provided?(Press y|Y for Yes, any other key for No) : y

By default, a MySQL installation has an anonymous user,

allowing anyone to log into MySQL without having to have

a user account created for them. This is intended only for

testing, and to make the installation go a bit smoother.

You should remove them before moving into a production

environment.

Remove anonymous users? (Press y|Y for Yes, any other key for No) : y

Success.

Normally, root should only be allowed to connect from

‘localhost’. This ensures that someone cannot guess at

the root password from the network.

Disallow root login remotely? (Press y|Y for Yes, any other key for No) : y

Success.

By default, MySQL comes with a database named ‘test’ that

anyone can access. This is also intended only for testing,

and should be removed before moving into a production

environment.

Remove test database and access to it? (Press y|Y for Yes, any other key for No) : y

 – Dropping test database…

Success.

 – Removing privileges on test database…

Success.

Reloading the privilege tables will ensure that all changes

made so far will take effect immediately.

Reload privilege tables now? (Press y|Y for Yes, any other key for No) : y

Success.

All done!

Note: If you don’t want some reason, do a “MySQL Secure Installation” then at least it’s very important to change the root user’s password

mysqladmin -u root password [your_password_here]

## Example ##

mysqladmin -u root password myownsecrectpass

  1. Connect to MySQL database (localhost) with password

mysql -u root -p

## OR ##

mysql -h localhost -u root -p

  1. Create Database, Create MySQL User and Enable Remote Connections to MySQL Database

This example uses following parameters:

  • DB_NAME = webdb
  • USER_NAME = webdb_user
  • REMOTE_IP = 10.0.15.25
  • PASSWORD = password123
  • PERMISSIONS = ALL

## CREATE DATABASE ##

mysql> CREATE DATABASE webdb;

## CREATE USER ##

mysql> CREATE USER ‘webdb_user’@’10.0.15.25’ IDENTIFIED BY ‘password123’;

## GRANT PERMISSIONS ##

mysql> GRANT ALL ON webdb.* TO ‘webdb_user’@’10.0.15.25’;

##  FLUSH PRIVILEGES, Tell the server to reload the grant tables  ##

mysql> FLUSH PRIVILEGES;

GRANT ALL PRIVILEGES ON *.* TO ‘root’@’%’ IDENTIFIED BY ‘Root@123’ WITH GRANT OPTION;

 

The 92% chunk asset optimisation problem nodejs Angular5

At times running Angular Application on an environment which is fresh can be daunting. We may end up with errors which are not in alignment towards the development. One such of a case is the 92% chunk asset optimization problem when running a build of the Angular application.

On AWS hooked up an ec2-instance with Ubuntu 16.04, every thing works fine until you run the command npm run build.

We are stuck in an exception, 92% chunk asset optimisation and the dist directory is not created. Though we end up looking information on the development stack, but at times it could also be related to the Operating System SWAP space.

If the SWAP space on the OS if short or not available, this error shows up and to go around the error follow the instructions to check and allocate swap space on ubuntu. Hopefully this should help in getting around the problem.

One of the easiest way of increasing the responsiveness of your server and guarding against out of memory errors in your applications is to add some swap space. Swap is an area on a hard drive that has been designated as a place where the operating system can temporarily store data that it can no longer hold in RAM.

Basically, this gives you the ability to increase the amount of information that your server can keep in its working “memory”, with some caveats. The space on the hard drive will be used mainly when space in RAM is no longer sufficient for data.

The information written to disk will be slower than information kept in RAM, but the operating system will prefer to keep running application data in memory and use swap for the older data. Overall, having swap space as a fall back for when your system’s RAM is depleted is a good safety net.

sudo swapon -s

Another command for the same thing

free -m

Before we do this, we should be aware of our current disk usage. We can get this information by typing:

df -h

Although there are many opinions about the appropriate size of a swap space, it really depends on your personal preferences and your application requirements. Generally, an amount equal to or double the amount of RAM on your system is a good starting point.

Since my system has 4 Gigabytes of RAM, and doubling that would take a significant chunk of my disk space that I’m not willing to part with, I will create a swap space of 4 Gigabytes to match my system’s RAM.

Creating SWAP File
sudo fallocate -l 4G /swapfile
Check by running the following command that the SWAP file was created and space was reserved
ls -lh /swapfile
Enable SWAP Fiile
sudo mkswap /swapfile
Verfiy Permissions are set correctly
ls -lh /swapfile
Now that our file is more secure, we can tell our system to set up the swap space by typing:
sudo swapon /swapfile

We can verify that the procedure was successful by checking whether our system reports swap space now:

sudo swapon -s
Filename                Type        Size    Used    Priority
/swapfile               file        4194300 0       -1

We have a new swap file here. We can use the free utility again to corroborate our findings:

free -m

https://www.digitalocean.com/community/tutorials/how-to-add-swap-on-ubuntu-14-04

Where are packages installed in Ubuntu : Debian Derivatives

Ubuntu, which is a Debian derivative, follows a very precise structure when installing packages. In other words, all software installed through the packaging tools, such as apt-get or synaptic, will put the stuff in the same locations. If you become familiar with these locations, you’ll always know where to find your stuff.

As a short cut, you can always open a tool like synaptic, find the installed package, and inspect the “properties”. Under properties, you’ll see a list of all installed files. Again, you can expect these to always follow the Debian/Ubuntu conventions; these are highly ordered Linux distributions. IN short, binaries will be in /usr/bin, or some other location on your path ( try ‘echo $PATH’ on the command line to see the possible locations ). Configuration is always in a subdirectory of /etc. And the “home” is typically in /usr/lib or /usr/share.

For instance installing maven,

sudo apt-get install maven

The Apt-get installation will install all the required files in the following folder structure

/usr/bin/mvn

/usr/share/maven2/

/etc/maven2

P.S The Maven configuration is store in /etc/maven2

Note, it’s not just apt-get that will do this, it’s any .deb package installer.

Uninstalling JAVA from Ubuntu

  1. Remove all the Java related packages (Sun, Oracle, OpenJDK, IcedTea plugins, GIJ):
    dpkg-query -W -f='${binary:Package}\n' | grep -E -e '^(ia32-)?(sun|oracle)-java' -e '^openjdk-' -e '^icedtea' -e '^(default|gcj)-j(re|dk)' -e '^gcj-(.*)-j(re|dk)' -e '^java-common' | xargs sudo apt-get -y remove
    sudo apt-get -y autoremove
  2. Purge config files:
    dpkg -l | grep ^rc | awk '{print($2)}' | xargs sudo apt-get -y purge
  3. Remove Java config and cache directory:
    sudo bash -c 'ls -d /home/*/.java' | xargs sudo rm -rf
  4. Remove manually installed JVMs:
    sudo rm -rf /usr/lib/jvm/*
  5. Remove Java entries, if there is still any, from the alternatives:
    for g in ControlPanel java java_vm javaws jcontrol jexec keytool mozilla-javaplugin.so orbd pack200 policytool rmid rmiregistry servertool tnameserv unpack200 appletviewer apt extcheck HtmlConverter idlj jar jarsigner javac javadoc javah javap jconsole jdb jhat jinfo jmap jps jrunscript jsadebugd jstack jstat jstatd native2ascii rmic schemagen serialver wsgen wsimport xjc xulrunner-1.9-javaplugin.so; do sudo update-alternatives --remove-all $g; done
  6. Search for possible remaining Java directories:
    sudo updatedb
    sudo locate -b '\pack200'

    If the command above produces any output like /path/to/jre1.6.0_34/bin/pack200 remove the directory that is parent of bin, like this: sudo rm -rf /path/to/jre1.6.0_34.

Installing Java

Step 1: Install Java 8 (JDK 8)
Open terminal and enter the following commands.
sudo add-apt-repository -y ppa:webupd8team/java
sudo apt-get update
echo debconf shared/accepted-oracle-license-v1-1 select true | sudo debconf-set-selections
echo debconf shared/accepted-oracle-license-v1-1 seen true | sudo debconf-set-selections
sudo apt-get -y install oracle-java8-installer

Setup JAVA Environment
sudo apt-get -y install oracle-java8-set-default

Step 2: Verify JAVA Version
After successfully installing oracle java using above step verify installed version using following command.
$ java -version
You see output similar to this.
java version “1.8.0_25”
Java(TM) SE Runtime Environment (build 1.8.0_25-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)

Installing JAVA on CentOS

How to Install JAVA 8 on CentOS/RHEL 7/6 and Fedora 28-23

Setting up a Reverse Proxy Server

Following our previous Process Manager for Nodejs Applications,  article, here we would get to see a walk through of how to set up a reverse proxy server for our nodejs application,

Now that our nodejs application is running, and listening on a private IP address, we need to set up a way where end users  access it. We will set up an Nginx web server as a reverse proxy. This tutorial will set up an Nginx server from scratch.

On the web server, let’s update the apt-get package lists with this command:

  • sudo apt-get update

Then install Nginx using apt-get:

  • sudo apt-get install nginx

Now open the default server block configuration file for editing:

  • sudo vi /etc/nginx/sites-available/default

Delete everything in the file and insert the following configuration. Be sure to substitute your own domain name for the server_name directive (or IP address if you don’t have a domain set up), and the app server private IP address for the APP_PRIVATE_IP_ADDRESS. Additionally, change the port (8080) if your application is set to listen on a different port:

/etc/nginx/sites-available/default
server {
    listen 80;

    server_name example.com;

    location / {
        proxy_pass http://APP_PRIVATE_IP_ADDRESS:8080;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

This configures the web server to respond to requests at its root. Assuming our server is available at example.com, accessing http://example.com/ via a web browser would send the request to the application server’s private IP address on port 8080, which would be received and replied to by the Node.js application.

You can add additional location blocks to the same server block to provide access to other applications on the same web server. For example, if you were also running another Node.js application on the app server on port 8081, you could add this location block to allow access to it via http://example.com/app2:

Nginx Configuration — Additional Locations
    location /app2 {
        proxy_pass http://APP_PRIVATE_IP_ADDRESS:8081;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }

Once you are done adding the location blocks for your applications, save and exit.

On the web server, restart Nginx:

  • sudo service nginx restart

Assuming that your Node.js application is running, and your application and Nginx configurations are correct, you should be able to access your application via the reverse proxy of the web server. Try it out by accessing your web server’s URL (its public IP address or domain name).

 

 

AWS Ubuntu SSH

To connect to AWS ensure to have the pem file at your disposal.

Check the pem file
ls *.pem
ls -la emailverifier-DC.pem
chmod -w emailverifier-DC.pem
ls -la emailverifier-DC.pem

Appropriate permissions on pem file
ls -la
ls -la emailverifier-DC.pem-~

chmod go-rw emailverifier-DC.pem
ls -la emailverifier-DC.pem
chmod -w emailverifier-DC.pem
pwd

chmod 400 emailverifier-dc.pem

-r——– 1 ubuntu ubuntu 1696 Jul 14 11:15 emailverifier.pem

Transfer a file from local system to remote system
scp /path/to/local/file username@hostname:/path/to/remote/file
scp -i emailverifier-DC.pem /home/user/Desktop/emails20170302.csv ubuntu@35.163.80.42:/home/ubuntu/EmailVerifier

Connect to Remote System ssh
ssh -i. emailverifier-DC.pem ubuntu@35.163.80.42

 

Transfer folder from local machine to AWS or To upload a file from your laptop to Amazon instance:

“scp” means “secure copy”, which can copy files between computers on a network. You can use this tool in a Terminal on a Unix/Linux/Mac system.

scp -i pems/Dataentrytool.pem -r datacaptive-data-entry/Data-entry-tool\ val/app/bin/ ec2-user@35.166.165.227:/home/ec2-user/Data-entry-tool/app/

scp -r user@your.server.example.com:/path/to/foo /home/user/Desktop/

Download a file from the remote server by issuing a cat command
scp username1@hostname1:/path/to/file username2@hostname2:/path/to/other/file
user@Sys208:~/Downloads$ ssh -i emailverifier-DC.pem  ubuntu@35.163.80.42 ‘cat ~/EmailVerifier/output_files/interesting_emails.csv’ > interesting_emails.csv

user@Sys208:~/Downloads$

********************************AWS*************************

1.Loading AWS session in putty

putty1

2.Upload SSH key for authentication(emailverify.ppk )

putty2

 

 

To download file using pem key from AWS to local machie

scp -i <key> root@<host>:<root/dir/subdir/file.pdf-whatever the path is> .

user@LT-201:~/Downloads$ scp -i “myKey.pem” bitnami@ec2-52-34-149-13.us-west-2.compute.amazonaws.com:/home/bitnami/mautic20170405.sql p

 

At times you would want to have files transferred locally from one machine to the other machine.

The following example would help you transfer nohup.out file from remote machine to the local machine, where the SSH port is different from 22 like 1337 as shown below

user@LT-201:  scp -P 1337 dcserver@192.168.0.58:/home/dcserver/nohup.out -p 1337 .

 

To transfer a file from local machine to remote machine on the LAN with different SSH port apart from 22, like 1337

user@Sys-201:~$ scp -P 1337 /home/user/Dataentrytool.pem dcserver@192.168.0.58:/home/dcserver/
dcserver@192.168.0.58’s password:
Dataentrytool.pem                             100% 1692     1.7KB/s   00:00

 

At times when transferring large files from the local system to the server, there can be timeout issues on the ssh terminal.

echo 'ServerAliveInterval 30' | sudo tee -a ~/.ssh/config
echo 'ServerAliveCountMax 1200' | sudo tee -a ~/.ssh/config

ServerAliveCountMax by default this is set to 3. 
Therefore once the ServerAliveInterval has sent 3 small packs of info to 
your server it will then automatically log out. Setting it to 1200 means 
this process will have to occur at least 1200 times. In short you should
be connected at least 30*1200 seconds (10 hours).


 

Uninstalling & Installing MYSQL on Ubuntu

Uninstalling MySql :

sudo apt-get purge mysql-server mysql-client mysql-common mysql-server-core-5.5 mysql-client-core-5.5
sudo rm -rf /etc/mysql /var/lib/mysql
sudo apt-get autoremove
sudo apt-get autoclean

Installation:

sudo apt-get update
sudo apt-get install mysql-server

Note:- Incase you encounter an ‘error:alternative path /etc/mysql/mysql.cnf’ then purge the unconfigured mysql using the

command:’sudo apt-get remove –purge mysql-\*’

& reinstall with the following command below

command:’sudo apt-get install mysql-server mysql-client’

Note: If mysql throws error “Access denied for user ‘root’@’localhost’ ” change the root user password by the following command

  • ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'password';
  • Where ‘password’ is the new password for the root user.

Ubuntu Ports, Process Commands

sudo netstat -peanut : Lists all the processes with ports

sudo netstat -peant | grep ":9300 "

user@user-HP-Compaq-8100-Elite-SFF-PC:/opt/logstash/bin$ sudo netstat -lanpt | grep LIST | grep java
tcp        0      0 127.0.0.1:63342         0.0.0.0:*               LISTEN      6493/java
tcp        0      0 0.0.0.0:39953           0.0.0.0:*               LISTEN      6493/java
tcp        0      0 127.0.0.1:6942          0.0.0.0:*               LISTEN      6493/java
tcp        0      0 127.0.0.1:35040         0.0.0.0:*               LISTEN      6493/java
tcp6       0      0 127.0.0.1:36237         :::*                    LISTEN      7028/java
tcp6       0      0 :::9200                 :::*                    LISTEN      15290/java
tcp6       0      0 :::9300                 :::*                    LISTEN      15290/java
tcp6       0      0 :::33527                :::*                    LISTEN      8160/java
tcp6       0      0 :::57852                :::*                    LISTEN      8160/java

user@user-HP-Compaq-8100-Elite-SFF-PC:/opt/logstash/bin$ ps auwwx | grep 15290
elastic+ 15290  0.7  4.0 4631044 323400 ?      Ssl  17:37   0:32 /usr/bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Delasticsearch -Des.foreground=yes -Des.path.home=/usr/share/elasticsearch -cp :/usr/share/elasticsearch/lib/elasticsearch-1.7.6.jar:/usr/share/elasticsearch/lib/*:/usr/share/elasticsearch/lib/sigar/* -Des.pidfile=/var/run/elasticsearch/elasticsearch.pid -Des.default.path.home=/usr/share/elasticsearch -Des.default.path.logs=/var/log/elasticsearch -Des.default.path.data=/var/lib/elasticsearch -Des.default.config=/etc/elasticsearch/elasticsearch.yml -Des.default.path.conf=/etc/elasticsearch org.elasticsearch.bootstrap.Elasticsearch
user     18111  0.0  0.0  21296   940 pts/19   S+   18:52   0:00 grep –color=auto 15290

 

To get a list of all users in Ubuntu

[ec2-user@ip-172-31-37-147 /]$ cut -d: -f1 /etc/passwd

 

nohup: No Hang Up

jobs -l : listing of no hups

ps -ef