Unix Commands/Scripts for '#Ls'

Unix Commands/Scripts for '#Ls' - 95 Commands/Scripts found

Sample 1. Maven Clean install without executing tests

mvn clean install -DskipTests

_{Like Feedback maven maven clean install maven install maven install without test maven build}

Sample 2. Finding any of the multiple texts strings in file

egrep 'Error|Exception|Debug' logfile.txt

_{Like Feedback egrep grep log grep}

Sample 3. Tail log to see only lines containing an error / exception in last 5000 lines

tail -5000 /opt/WebSphere6/AppServer/profiles/Viva/logs/VivaWebClusterMemberPsc9800/SystemOut.log | grep -i "FileNotFoundException"

_{Like Feedback tail tail logs check logs for errors grep grep -i grep ignore case}

Sample 4. Do a maven clean install without tests and only display Error lines

mvn clean install -DskipTests | grep "ERROR"

_{Like Feedback maven maven install maven build maven clean install}

Sample 5. Get lines containing any of the multiple errors / exceptions in running logs

tail -f /opt/WebSphere/AppServer/profiles/application/logs/SystemOut.log | egrep "(WSWS3713E|WSWS3734W|WSVR0605W|javax.net.ssl.SSLHandshakeException|ThreadMonitor)"

_{Like Feedback grep grep logs tail tail -f egrep}

Sample 6. Count a particular word ( error ) in a file

grep -c "Error" logfile.txt

_{Like Feedback grep grep log grep -c}

Sample 7. count number of error / exception occurences in a file.

sed -n '/ERROR/,/EST/p' /opt/WebSphere/AppServer/profiles/application/logs/SystemOut.log | grep "LogicBlockSetupException" | wc -l

_{Like Feedback sed grep word count wc wc -l}

Sample 8. Switch Java version in Ubuntu

sudo update-alternatives --config java

_{Like Feedback switch java version sudo update-alternatives}

Sample 9. Get few lines before and after a particular dependency within mvn depedency tree ( 3 lines before and 1 line after )

mvn dependency:tree | grep -B 3 -A 1 log4j-slf4j

_{Like Feedback}

Sample 10. Get a list of files within AWS S3 bucket

aws s3 ls s3://<bucketName>



Example -



aws s3 ls s3://myBucket/bucket

_{Like Feedback aws s3 aws storage amazon cloud storage amazon s3 list files within aws s3 aws storage amazon cloud storage amazon s3}

Sample 11. Start an EC2 instance using AWS CLI

aws ec2 start-instances --instance-id=<EC2_INSTANCE_ID>

_{Like Feedback}

Sample 12. Commands and Steps for resolving Merge conflict in Git

git fetch ( it will only bring the remote code to local but won't attempt to merge )

git status ( will show the merge conflicts )

Fix the Merge conflicts

git add .

git commit 

git push

_{Like Feedback}

Sample 13. Git - your branch is ahead by 1 commit error. Move Head to previous commit ( Will reset the last change )

git reset --hard HEAD~

_{Like Feedback git}

Sample 14. Finding relevant word and excluding irrelevant word

grep xception logfile.txt | grep -v ERROR

_{Like Feedback grep grep log grep -v}

Sample 15. Finding text within .gz (zipped) file without unzipping them

zgrep -i Error *.gz

_{Like Feedback grep zgrep zgrep -i grep within zipped file}

Sample 16. find occurences of a particular error in last n days

find /opt/WebSphere/AppServer/profiles/application/logs/ -iname "SystemOut*" -mtime -7 -exec zgrep "FileNotFoundException" {} ; >> logAnalysis.txt

_{Like Feedback find zgrep -iname 0mtime >>}

Sample 17. Shell Script for Log4j Log Analysis and exception reporting

#!/bin/ksh



EMAIL_SUBJECT="Exceptions-Log"

EMAIL_TO="xyz@yahoo.com"

`grep "xception" Out.log >> /home/xyz/test1`

`cat /home/xyz/test1 | sed -n 's/.* ([^ ]*xception[^ ]*) .*/1/p' | awk '!x[$0]++' >> /home/xyz/test2`



`rm test3`



while read line

do

lineNum=`sed -n "/$line/,/EST/{=;q;}" Out.log`

let bl=$lineNum-5

let el=$lineNum+15

echo "************************************************* 

 $line 

 ************************************************** 



" >> test3

`sed -n "$bl,$el p" Out.log >> test3`

echo "





" >> test3

done < "/home/xyz/test2"



`cat test3 | /bin/mail -s $EMAIL_SUBJECT $EMAIL_TO`

_{Like Feedback script exception reporting script log4j analysis script awk}

Sample 18. Log Monitoring Shell Script - email upon errors

#!/bin/ksh

# Set the config variables



# *************************************************Configuration********************************************************

logFileName="Out.log"

errorList="WSWS3713E|WSWS3734W|WSVR0605W|javax.net.ssl.SSLHandshakeException|ThreadMonitor"

EMAIL_SUBJECT="Application ERROR"

EMAIL_TO="viva@viva.com"

# **********************************************************************************************************************



logFilepath=""



# Set the Log File path



if [ `hostname` = cpc2600 ]

then

logFilePath="/opt/WebSphere6/AppServer/profiles/applicationcpc2600/logs/"

elif [ `hostname` = cpc2601 ]

then

logFilePath="/opt/WebSphere6/AppServer/profiles/applicationcpc2600/logs "

elif [ `hostname` = psc2800 ]

then

logFilePath="/opt/WebSphere6/AppServer/profiles/applicationpsc2800/logs "

elif [ `hostname` = psc2801 ]

then

logFilePath="/opt/WebSphere6/AppServer/profiles/applicationpsc2801/logs "

fi



if [ ! -s $logFilePath/$logFileName ]; then echo "ERROR- Log File Not Found , Please set the config properly"

exit

fi



# Get the first 30 characters of the first line linestart=$(awk 'NR>1{exit} ;1' $logFilePath/$logFileName | cut -c1-30)



lineend=""



# Never ending loop that will parse the Out.log file every 5 sec



while true ; do



# get the last line of file , till which we need to parse the log in this iteration lineend=$(awk 'END{print}'

$logFilePath/$logFileName | cut -c1-30)



# if log file not found , Do nothing and wait for the next iteration if [ ! -s $logFilePath/$logFileName ];

then echo "Log File Not Found .. Waiting for the next iteration ..."

fi



# error checking , in case we dont find the linestart , parse the whole file grep "$linestart"

$logFilePath/$logFileName if [ $? != 0 ] then

echo "cat $logFilePath/$logFileName | egrep $errorList | /usr/sbin/sendmail -s $EMAIL_SUBJECT $EMAIL_TO"

cat $logFilePath/$logFileName | egrep "$errorList" | /usr/sbin/sendmail -s $EMAIL_SUBJECT $EMAIL_TO



else



#parse the log file from linestart to lineend for errors



echo 'awk "/$linestart/,/$lineend/" $logFilePath/$logFileName | egrep "$errorList" | /usr/sbin/sendmail -s $EMAIL_SUBJECT $EMAIL_TO'



awk "/$linestart/,/$lineend/" $logFilePath/$logFileName | egrep "$errorList" | /usr/sbin/sendmail -s $EMAIL_SUBJECT $EMAIL_TO #set the last line as the first line for next iteration linestart=$lineend fi



#set the last line as the first line for next iteration linestart=$lineend



sleep 5



done

_{Like Feedback script shell script to report errors in logs}

Sample 19. Get all Errors in a log file for last n days in a seperate error file

find /LogFilesfolder/ -iname "SystemOut*" -mtime -7 -exec zgrep "| ERROR |" {} ; >> logReport.txt

_{Like Feedback find -iname -mtime zgrep find errors in log file}

Sample 20. Clone a repository / checkout project in Git

git clone <repository_url>

_{Like Feedback}

Sample 21. How to reload .bash_profile in ubuntu

source ~/.bash_profile

_{Like Feedback}

Sample 22. Install ruby on ubuntu

sudo apt-get install ruby-full

_{Like Feedback}

Sample 23. Install AWS CLI on Linux / ubuntu

sudo apt-get install awscli

_{Like Feedback}

Sample 24. Run df command ( check disk space usage ) using AWS CLI on AWS EC2 instance

Usage -



aws ssm send-command --document-name "AWS-RunPowerShellScript" --parameters commands=["df -a"] --targets "Key=instanceids,Values=<instanceId>"



Example -



aws ssm send-command --document-name "AWS-RunPowerShellScript" --parameters commands=["df -a"] --targets "Key=instanceids,Values= i-0981d8456ffd18wcb"

_{Like Feedback aws cli aws ssm aws ssm send-command check disk space usage on AWS instanc}

Sample 25. Get Summary / List of files recursively from an S3 bucket

aws s3 ls s3://mybucket --recursive --human-readable --summarize

_{Like Feedback amazon aws s3 aws storage amazon cloud storage amazon s3}

Sample 26. Download S3 public file

wget <S3 URL>

_{Like Feedback Download s3 file s3 file amazon s3}

Sample 27. Get Elastic Beanstalk / EC2 instance information using instance name

aws ec2 describe-instances --output text --filters "Name=tag:Name,Values=<ELASTIC_BEANSTALK_INSTANCE_NAME>"

_{Like Feedback AWS Amazon AWS AWS CLI describe-instances Amazon EC2}

Sample 28. Get List of AWS Configuration

aws configure list

_{Like Feedback}

Sample 29. Build angular project

 ng build

_{Like Feedback}

Sample 30. Display the file names that matches the given pattern

grep -l ERROR *.log

_{Like Feedback grep}

Sample 31. Get Error Snippets in running logs

tail -f /opt/WebSphere/AppServer/profiles/application/logs/SystemOut.log | sed -n '/ERROR/,/EST/p'

_{Like Feedback grep grep logs tail tail -f sed sed -n}

Sample 32. Report the file size of all files bigger than 2 mb and older than 30 days

find . -type f -size +4096 -atime +30 -exec du -sk '{}' ;

_{Like Feedback find du -exec find files}

Sample 33. Shell Scripts for Automating System Monitoring Task

#!/bin/ksh



errorSnippet=''



# ********************************************************Configuration***************************************************************

homeBench='90'

VivaBench='90'

rootBench='90'

appHomeBench='90'

idleBench='95'

logsOldBench='15'

memUsageBench='2500'

avgLoadBench='5'

EMAIL_SUBJECT="Server Health Check Report for $(hostname)"

EMAIL_TO="test@test.com"

# ************************************************************************************************************************************



dfHome=`df | sed -n '/ /home$/s/.* ([0-9][0-9]*)%.*/1/p'`

dfViva=`df | sed -n '/ /apphome/Viva$/s/.* ([0-9][0-9]*)%.*/1/p'`

dfRoot=`df | sed -n '/ /$/s/.* ([0-9][0-9]*)%.*/1/p'`

dfApphome=`df | sed -n '/ /apphome$/s/.* ([0-9][0-9]*)%.*/1/p'`

dfLogsOld=`df | sed -n '/ /localvg-logsOld$/s/.* ([0-9][0-9]*)%.*/1/p'`

memUsage=`sar -q 1 | tail -1 | awk '{ print "" $3}' | sed 's/%//g'`

avgLoad=`uptime | awk -F "$FTEXT" '{ print $2 }' | cut -d, -f3`

iostatIdle=`iostat | awk '{print $5}' | awk 'NR==4' | cut -d '.' -f1`



if [[ $dfHome -gt $homeBench ]] then

errorSnippet="Disk Usage for /home exceedeed the benchmark, Its $dfHome now";

fi

if [[ $dfViva -gt $VivaBench ]] then

errorSnippet="$errorSnippet 

 Disk Usage for /apphome/Viva exceedeed the benchmark, Its $dfViva now";

fi

if [[ $dfRoot -gt $rootBench ]] then

errorSnippet="$errorSnippet 

 Disk Usage for /(root) exceedeed the benchmark, Its $dfRoot now";

fi

if [[ $dfRoot -gt $rootBench ]] then

errorSnippet="$errorSnippet 

 Disk Usage for /(root) exceedeed the benchmark, Its $dfRoot now";

fi

if [[ $dfApphome -gt $appHomeBench ]] then

errorSnippet="$errorSnippet 

 Disk Usage for /apphome exceedeed the benchmark, Its $dfApphome now";

fi

if [[ $dfLogsOld -gt $logsOldBench ]] then

errorSnippet="$errorSnippet 

 Disk Usage for logs old exceedeed the benchmark, Its $dfLogsOld now";

fi

if [[ $iostatIdle -gt $idleBench ]] then

errorSnippet="$errorSnippet 

 Iostat idle exceedeed the benchmark, Its $iostatIdle now";

fi

if [[ $memUsage -gt $memUsageBench ]] then

errorSnippet="$errorSnippet 

 Memory Usage exceedeed the benchmark, Its $memUsage now";

fi

if [[ $avgLoad -gt $avgLoadBench ]] then

errorSnippet="$errorSnippet 

 15 minute Average Load exceedeed the benchmark, Its $avgLoad now";

fi



print $errorSnippet

if [ "$errorSnippet" != "" ]; then

`echo errorSnippet | /bin/mail -s $EMAIL_SUBJECT $EMAIL_TO`

fi

_{Like Feedback script to automate monitoring task df sed sar tail awk iostat shell script if block print /bin/mail}

Sample 34. Print last field using awk

awk { print $NF }

_{Like Feedback awk}

Sample 35. Count words in a File

grep "Text" file.txt | wc -l

_{Like Feedback count words in a file grep wc word count}

Sample 36. Build Maven project offline

mvn o package

_{Like Feedback maven}

Sample 37. Count number of lines in a file

wc -l

_{Like Feedback}

Sample 38. Find text within files of a directory recursively

grep -R "<text>"

_{Like Feedback}

Sample 39. Writes a single data record into an Amazon Kinesis stream

aws kinesis put-record --stream-name <STREAM_NAME> --partition-key <PARTITION_KEY> --data <DATA>

_{Like Feedback Amazon AWS AWS Amazon Kinesis Stream aws kinesis put-record aws kinesis put single record in amazon Kinesis}

Sample 40. Get all Tag information for EC2 instance using instance name

aws ec2 describe-instances --output text --filters "Name=tag:Name,Values=<ELASTIC_BEANSTALK_INSTANCE_NAME>" | grep "TAGS"

_{Like Feedback AWS Amazon AWS AWS CLI describe-instances Amazon EC2}

Sample 41. Get All attributes for a key

hgetall <KEY>

_{Like Feedback Redis}

Sample 42. SSH into EC2 instance using pem ( key ) file

ssh -I <KEY_FILE_NAME> <USER_NAME>@<PUBLIC_IP>

_{Like Feedback}

Sample 43. Moving file from staging to head

git commit <file_name>

_{Like Feedback git}

Sample 44. Build Docker Image

docker build <FILE_OR_FOLDER>

_{Like Feedback}

Sample 45. Shell Script to continuously monitor the state(up/down) of application and send email if its down

while true ; do

curl websiteaddress.com | grep -q "Down for Maintenance"

if [ $? -eq 0 ] ; then

echo "Website is Down" | mail -s "Website is down for maintenance" email@address.com

; fi

sleep 20

done

_{Like Feedback script to monitor logs curl sleep while loop if block}

Sample 46. Grant most liberal permissions on the file or directory for all ( self , group and others )

chmod 777 file_name

_{Like Feedback chmod change permission change permission on file grant most liberal permission grant most lenient permissions}

Sample 47. Get the Maven Dependency Tree of a project in a separate file

mvn dependency:tree >> file.txt

_{Like Feedback maven maven dependency maven dependency tree}

Sample 48. Get fields matching a particular string using awk

awk '/xception/ { print $1 }' source.txt >> destination.txt

_{Like Feedback awk}

Sample 49. How to display hidden files in unix

^{Ashish Singh
ashishde89@gmail.com}

ls -a

_{Like Feedback display hidden files listing all files ls}

Sample 50. Get Maven Dependency using command line get

mvn dependency:get -Dartifact=org.springframework:spring-context:5.0.0.BUILD-SNAPSHOT



i.e 



mvn dependency:get -Dartifact=<group Id>:<artifact Id>:<version>

_{Like Feedback Get Maven Dependency using command line get}

Sample 51. Run Jetty server through command line using maven

mvn jetty:run

_{Like Feedback run jetty}

Sample 52. Check JAVA_HOME value

echo $JAVA_HOME

_{Like Feedback JAVA_HOME}

Sample 53. Default Location of Maven Repository

~/m2./repository

_{Like Feedback maven}

Sample 54. Recursively create directories

mkdir -p /home/newParentDir/newDir

_{Like Feedback}

Sample 55. Location of .bash_profile in ubuntu

Its in your Home directory

_{Like Feedback}

Sample 56. Install AWS CLI ( Amazon Web Service Command Line Interface ) using brew

brew install awscli

_{Like Feedback}

Sample 57. Install MySQL using brew

brew install mysql

_{Like Feedback}

Sample 58. Install redis using brew

brew install redis

_{Like Feedback}

Sample 59. Execute class with main method using maven

Execute it from the folder with pom file



mvn exec:java -Dexec.mainClass="com.*.*"

_{Like Feedback}

Sample 60. Load Data from CSV into MySql Table

USAGE



LOAD DATA LOCAL INFILE '<CSV_FILE_NAME>' INTO TABLE <TABLE_NAME> FIELDS TERMINATED BY '<FIELD_SEPARATOR>';



EXAMPLE



LOAD DATA LOCAL INFILE 'xyz.csv' INTO TABLE EMPLOYEE FIELDS TERMINATED BY ',';

_{Like Feedback load csv into mysql table csv to mysql}

Sample 61. Get current Date in a specified format ( year, month,date,hour and minute) within shell script.

currentDate=$(date "+%Y-%m-%dT%H:%M")

_{Like Feedback current date current date in specific format}

Sample 62. Get information about a particular replication group

aws describe-replication-groups --replication-group-id <REPLICATION_GROUP_ID> --output text

_{Like Feedback AWS Amazon AWS AWS CLI describe-replication-groups}

Sample 63. Get Private IP address of a EC2 / Beanstalk instance using AWS CLI

aws ec2 describe-instances --output text --filters "Name=tag:Name,Values=<ELASTIC_BEANSTALK_INSTANCE_NAME>" | grep "PRIVATEIPADDRESSES"

_{Like Feedback AWS Amazon AWS AWS CLI describe-instances Amazon EC2}

Sample 64. Remove all folders and subfolders recursively and forcibly

rm -rf *

_{Like Feedback}

Sample 65. How to find lines in a file "File" containing "some text" ?

grep "some Text" File

_{Like Feedback}

Sample 66. Command to list of jdk modules

java --list-modules

_{Like Feedback java 9 java 9 module}

Sample 67. Check if AWS is installed

which aws

_{Like Feedback}

Sample 68. Check if AWS has been configured properly

aws configure list | egrep "access_key|secret_key|region" | wc -l



Count of 3 means configured correctly

_{Like Feedback}

Sample 69. Get Elastic Cache Information

aws elasticache describe-replication-groups --query 'ReplicationGroups[?ReplicationGroupId==`<ELASTIC_CACHE_IDENTIFIER>`].NodeGroups[0].{ip:PrimaryEndpoint.Address}' --output text

_{Like Feedback}

Sample 70. Stop an EC2 instance through AWS CLI

aws ec2 stop-instances --instance-id=<EC2_INSTANCE_ID>

_{Like Feedback}

Sample 71. Instructing maven to set up files ( class path, project ) within eclipse

mvn eclipse:eclipse

_{Like Feedback maven eclipse}

Sample 72. Instructing maven to set up files ( class path, project ) within idea IntelliJ

mvn idea:idea

_{Like Feedback maven IntelliJ}

Sample 73. Set a Redis field value using key and field name

or Add a new Field to a Redis Key

HSET <Key> <field> <value>

_{Like Feedback Redis AWS}

Sample 74. Connect to Redis using Redis CLI

redis-cli -c -h <Redis End Point> -p <Port>

_{Like Feedback Redis aws}

Sample 75. Run a program / command with security privileges of root / others

su



or



su <username>



or



sudo su <username>

_{Like Feedback}

Sample 76. Running angular development server

ng serve

_{Like Feedback}

Sample 77. Create mew angular component

ng generate component <component_name>

_{Like Feedback}

Sample 78. Create new angular project using angular CLI

ng new <project_name>

_{Like Feedback}

Sample 79. Create new angular project using angular CLI with routing enabled

ng new <project_name> --routing

_{Like Feedback}

Sample 80. Moving a file to staging

git add

_{Like Feedback git}

Sample 81. Moving file from Head on local to remote ?

git push <file_name>

_{Like Feedback git}

Sample 82. Does git commit , commits the file to remote ?

_{Like Feedback git}

Sample 83. Setting the remote project for the Git initialized local project

git remote add origin <git_repo_url>

_{Like Feedback git}

Sample 84. Merging remote changes to local

git pull

_{Like Feedback git}

Sample 85. Add all files to the index for commit and push in git

git add .

_{Like Feedback}

Sample 86. Steps in Git push from local to remote

git add .

git commit -m "message"

git push

_{Like Feedback}

Sample 87. Installing angular Redux module

npm install redux @angular-redux/store

_{Like Feedback angular redux}

Sample 88. Put a folder recursively using sftp

sftp <user_name>@<machine_address> 



put - r <folder_path>

_{Like Feedback sftp}

Sample 89. Git - Remove a Folder forced and recursively from the index

git rm -rf <folder_name>

_{Like Feedback}

Sample 90. Git - Add only java files to the index

git add *.java

_{Like Feedback}

Sample 91. Load Docker image locally

sudo docker load -i <LOCAL_IMAGE_FILE>

_{Like Feedback}

Sample 92. shim error: docker-runc not installed on system

Run -



sudo ln -s /usr/libexec/docker/docker-runc-current /usr/bin/docker-runc

_{Like Feedback}

Sample 93. Could not find module "@angular-devkit/build-angular" from

npm install --save-dev @angular-devkit/build-angular

_{Like Feedback}

Sample 94. Install RxJs

npm install rxjs

_{Like Feedback}

Sample 95. Install rxjs-compat

npm install --save rxjs-compat

_{Like Feedback}

Search Unix Commands/Scripts