Share

Search Unix Commands/Scripts


  Help us in improving the repository. Add new commands/scripts through 'Submit Commands/Scripts ' link.





Unix Commands/Scripts for '#Ls' - 86 Commands/Scripts found

 Sample 1. Finding any of the multiple texts strings in file

egrep 'Error|Exception|Debug' logfile.txt

   Like      Feedback     egrep   grep log  grep


 Sample 2. Tail log to see only lines containing an error / exception in last 5000 lines

tail -5000 /opt/WebSphere6/AppServer/profiles/Viva/logs/VivaWebClusterMemberPsc9800/SystemOut.log | grep -i "FileNotFoundException"

   Like      Feedback     tail  tail logs  check logs for errors  grep  grep -i  grep ignore case


 Sample 3. Get lines containing any of the multiple errors / exceptions in running logs

tail -f /opt/WebSphere/AppServer/profiles/application/logs/SystemOut.log | egrep "(WSWS3713E|WSWS3734W|WSVR0605W|javax.net.ssl.SSLHandshakeException|ThreadMonitor)"

   Like      Feedback     grep  grep logs  tail  tail -f  egrep


 Sample 4. Maven Clean install without executing tests

mvn clean install -DskipTests

   Like      Feedback     maven  maven clean install  maven install  maven install without test  maven build


Subscribe to Java News and Posts. Get latest updates and posts on Java from Buggybread.com
Enter your email address:
Delivered by FeedBurner
 Sample 5. Do a maven clean install without tests and only display Error lines

mvn clean install -DskipTests | grep "ERROR"

   Like      Feedback     maven  maven install  maven build  maven clean install


 Sample 6. find occurences of a particular error in last n days

find /opt/WebSphere/AppServer/profiles/application/logs/ -iname "SystemOut*" -mtime -7 -exec zgrep "FileNotFoundException" {} ; >> logAnalysis.txt

   Like      Feedback     find  zgrep  -iname  0mtime  >>


 Sample 7. Shell Script for Log4j Log Analysis and exception reporting

#!/bin/ksh

EMAIL_SUBJECT="Exceptions-Log"
EMAIL_TO="xyz@yahoo.com"
`grep "xception" Out.log >> /home/xyz/test1`
`cat /home/xyz/test1 | sed -n 's/.* ([^ ]*xception[^ ]*) .*/1/p' | awk '!x[$0]++' >> /home/xyz/test2`

`rm test3`

while read line
do
lineNum=`sed -n "/$line/,/EST/{=;q;}" Out.log`
let bl=$lineNum-5
let el=$lineNum+15
echo "*************************************************
$line
**************************************************

" >> test3
`sed -n "$bl,$el p" Out.log >> test3`
echo "


" >> test3
done < "/home/xyz/test2"

`cat test3 | /bin/mail -s $EMAIL_SUBJECT $EMAIL_TO`

   Like      Feedback     script  exception reporting script  log4j analysis script  awk


 Sample 8. Switch Java version in Ubuntu

sudo update-alternatives --config java

   Like      Feedback     switch java version   sudo update-alternatives


 Sample 9. Finding relevant word and excluding irrelevant word

grep xception logfile.txt | grep -v ERROR

   Like      Feedback     grep   grep log  grep -v


Subscribe to Java News and Posts. Get latest updates and posts on Java from Buggybread.com
Enter your email address:
Delivered by FeedBurner
 Sample 10. Count a particular word ( error ) in a file

grep -c "Error" logfile.txt

   Like      Feedback     grep   grep log  grep -c


 Sample 11. Finding text within .gz (zipped) file without unzipping them

zgrep -i Error *.gz

   Like      Feedback     grep  zgrep  zgrep -i  grep within zipped file


 Sample 12. Display the file names that matches the given pattern

grep -l ERROR *.log

   Like      Feedback     grep


 Sample 13. Get Error Snippets in running logs

tail -f /opt/WebSphere/AppServer/profiles/application/logs/SystemOut.log | sed -n '/ERROR/,/EST/p'

   Like      Feedback     grep  grep logs  tail  tail -f  sed  sed -n


 Sample 14. count number of error / exception occurences in a file.

sed -n '/ERROR/,/EST/p' /opt/WebSphere/AppServer/profiles/application/logs/SystemOut.log | grep "LogicBlockSetupException" | wc -l

   Like      Feedback     sed  grep  word count  wc  wc -l


Subscribe to Java News and Posts. Get latest updates and posts on Java from Buggybread.com
Enter your email address:
Delivered by FeedBurner
 Sample 15. Report the file size of all files bigger than 2 mb and older than 30 days

find . -type f -size +4096 -atime +30 -exec du -sk '{}' ;

   Like      Feedback     find   du   -exec  find files


 Sample 16. Log Monitoring Shell Script - email upon errors

#!/bin/ksh
# Set the config variables

# *************************************************Configuration********************************************************
logFileName="Out.log"
errorList="WSWS3713E|WSWS3734W|WSVR0605W|javax.net.ssl.SSLHandshakeException|ThreadMonitor"
EMAIL_SUBJECT="Application ERROR"
EMAIL_TO="viva@viva.com"
# **********************************************************************************************************************

logFilepath=""

# Set the Log File path

if [ `hostname` = cpc2600 ]
then
logFilePath="/opt/WebSphere6/AppServer/profiles/applicationcpc2600/logs/"
elif [ `hostname` = cpc2601 ]
then
logFilePath="/opt/WebSphere6/AppServer/profiles/applicationcpc2600/logs "
elif [ `hostname` = psc2800 ]
then
logFilePath="/opt/WebSphere6/AppServer/profiles/applicationpsc2800/logs "
elif [ `hostname` = psc2801 ]
then
logFilePath="/opt/WebSphere6/AppServer/profiles/applicationpsc2801/logs "
fi

if [ ! -s $logFilePath/$logFileName ]; then echo "ERROR- Log File Not Found , Please set the config properly"
exit
fi

# Get the first 30 characters of the first line linestart=$(awk 'NR>1{exit} ;1' $logFilePath/$logFileName | cut -c1-30)

lineend=""

# Never ending loop that will parse the Out.log file every 5 sec

while true ; do

# get the last line of file , till which we need to parse the log in this iteration lineend=$(awk 'END{print}'
$logFilePath/$logFileName | cut -c1-30)

# if log file not found , Do nothing and wait for the next iteration if [ ! -s $logFilePath/$logFileName ];
then echo "Log File Not Found .. Waiting for the next iteration ..."
fi

# error checking , in case we dont find the linestart , parse the whole file grep "$linestart"
$logFilePath/$logFileName if [ $? != 0 ] then
echo "cat $logFilePath/$logFileName | egrep $errorList | /usr/sbin/sendmail -s $EMAIL_SUBJECT $EMAIL_TO"
cat $logFilePath/$logFileName | egrep "$errorList" | /usr/sbin/sendmail -s $EMAIL_SUBJECT $EMAIL_TO

else

#parse the log file from linestart to lineend for errors

echo 'awk "/$linestart/,/$lineend/" $logFilePath/$logFileName | egrep "$errorList" | /usr/sbin/sendmail -s $EMAIL_SUBJECT $EMAIL_TO'

awk "/$linestart/,/$lineend/" $logFilePath/$logFileName | egrep "$errorList" | /usr/sbin/sendmail -s $EMAIL_SUBJECT $EMAIL_TO #set the last line as the first line for next iteration linestart=$lineend fi

#set the last line as the first line for next iteration linestart=$lineend

sleep 5

done

   Like      Feedback     script  shell script to report errors in logs


 Sample 17. Shell Scripts for Automating System Monitoring Task

#!/bin/ksh

errorSnippet=''

# ********************************************************Configuration***************************************************************
homeBench='90'
VivaBench='90'
rootBench='90'
appHomeBench='90'
idleBench='95'
logsOldBench='15'
memUsageBench='2500'
avgLoadBench='5'
EMAIL_SUBJECT="Server Health Check Report for $(hostname)"
EMAIL_TO="test@test.com"
# ************************************************************************************************************************************

dfHome=`df | sed -n '/ /home$/s/.* ([0-9][0-9]*)%.*/1/p'`
dfViva=`df | sed -n '/ /apphome/Viva$/s/.* ([0-9][0-9]*)%.*/1/p'`
dfRoot=`df | sed -n '/ /$/s/.* ([0-9][0-9]*)%.*/1/p'`
dfApphome=`df | sed -n '/ /apphome$/s/.* ([0-9][0-9]*)%.*/1/p'`
dfLogsOld=`df | sed -n '/ /localvg-logsOld$/s/.* ([0-9][0-9]*)%.*/1/p'`
memUsage=`sar -q 1 | tail -1 | awk '{ print "" $3}' | sed 's/%//g'`
avgLoad=`uptime | awk -F "$FTEXT" '{ print $2 }' | cut -d, -f3`
iostatIdle=`iostat | awk '{print $5}' | awk 'NR==4' | cut -d '.' -f1`

if [[ $dfHome -gt $homeBench ]] then
errorSnippet="Disk Usage for /home exceedeed the benchmark, Its $dfHome now";
fi
if [[ $dfViva -gt $VivaBench ]] then
errorSnippet="$errorSnippet
Disk Usage for /apphome/Viva exceedeed the benchmark, Its $dfViva now";
fi
if [[ $dfRoot -gt $rootBench ]] then
errorSnippet="$errorSnippet
Disk Usage for /(root) exceedeed the benchmark, Its $dfRoot now";
fi
if [[ $dfRoot -gt $rootBench ]] then
errorSnippet="$errorSnippet
Disk Usage for /(root) exceedeed the benchmark, Its $dfRoot now";
fi
if [[ $dfApphome -gt $appHomeBench ]] then
errorSnippet="$errorSnippet
Disk Usage for /apphome exceedeed the benchmark, Its $dfApphome now";
fi
if [[ $dfLogsOld -gt $logsOldBench ]] then
errorSnippet="$errorSnippet
Disk Usage for logs old exceedeed the benchmark, Its $dfLogsOld now";
fi
if [[ $iostatIdle -gt $idleBench ]] then
errorSnippet="$errorSnippet
Iostat idle exceedeed the benchmark, Its $iostatIdle now";
fi
if [[ $memUsage -gt $memUsageBench ]] then
errorSnippet="$errorSnippet
Memory Usage exceedeed the benchmark, Its $memUsage now";
fi
if [[ $avgLoad -gt $avgLoadBench ]] then
errorSnippet="$errorSnippet
15 minute Average Load exceedeed the benchmark, Its $avgLoad now";
fi

print $errorSnippet
if [ "$errorSnippet" != "" ]; then
`echo errorSnippet | /bin/mail -s $EMAIL_SUBJECT $EMAIL_TO`
fi

   Like      Feedback     script to automate monitoring task  df  sed  sar  tail  awk  iostat  shell script if block  print  /bin/mail


 Sample 18. Shell Script to continuously monitor the state(up/down) of application and send email if its down

while true ; do
curl websiteaddress.com | grep -q "Down for Maintenance"
if [ $? -eq 0 ] ; then
echo "Website is Down" | mail -s "Website is down for maintenance" email@address.com
; fi
sleep 20
done

   Like      Feedback     script to monitor logs  curl  sleep  while loop  if block


 Sample 19. Grant most liberal permissions on the file or directory for all ( self , group and others )

chmod 777 file_name

   Like      Feedback     chmod  change permission   change permission on file  grant most liberal permission  grant most lenient permissions


Subscribe to Java News and Posts. Get latest updates and posts on Java from Buggybread.com
Enter your email address:
Delivered by FeedBurner
 Sample 20. Get the Maven Dependency Tree of a project in a separate file

mvn dependency:tree >> file.txt

   Like      Feedback     maven  maven dependency  maven dependency tree


 Sample 21. Get fields matching a particular string using awk

awk '/xception/ { print $1 }' source.txt >> destination.txt

   Like      Feedback     awk


 Sample 22. Print last field using awk

awk { print $NF }

   Like      Feedback     awk


 Sample 23. Count words in a File

grep "Text" file.txt | wc -l

   Like      Feedback     count words in a file  grep  wc  word count


 Sample 24. Get all Errors in a log file for last n days in a seperate error file

find /LogFilesfolder/ -iname "SystemOut*" -mtime -7 -exec zgrep "| ERROR |" {} ; >> logReport.txt

   Like      Feedback     find  -iname  -mtime  zgrep  find errors in log file


Subscribe to Java News and Posts. Get latest updates and posts on Java from Buggybread.com
Enter your email address:
Delivered by FeedBurner
 Sample 25. How to display hidden files in unix
Ashish Singh
ashishde89@gmail.com
ls -a

   Like      Feedback     display hidden files   listing all files   ls


 Sample 26. Get Maven Dependency using command line get

mvn dependency:get -Dartifact=org.springframework:spring-context:5.0.0.BUILD-SNAPSHOT

i.e

mvn dependency:get -Dartifact=<group Id>:<artifact Id>:<version>

   Like      Feedback     Get Maven Dependency using command line get


 Sample 27. Run Jetty server through command line using maven

mvn jetty:run

   Like      Feedback     run jetty


 Sample 28. Check JAVA_HOME value

echo $JAVA_HOME

   Like      Feedback     JAVA_HOME


 Sample 29. Default Location of Maven Repository

~/m2./repository

   Like      Feedback     maven


Subscribe to Java News and Posts. Get latest updates and posts on Java from Buggybread.com
Enter your email address:
Delivered by FeedBurner
 Sample 30. Build Maven project offline

mvn o package

   Like      Feedback     maven


 Sample 31. Get few lines before and after a particular dependency within mvn depedency tree ( 3 lines before and 1 line after )

mvn dependency:tree | grep -B 3 -A 1 log4j-slf4j

   Like      Feedback     


 Sample 32. Count number of lines in a file

wc -l 

   Like      Feedback     


 Sample 33. Recursively create directories

mkdir -p /home/newParentDir/newDir

   Like      Feedback     


 Sample 34. Find text within files of a directory recursively

grep -R "<text>"

   Like      Feedback     


Subscribe to Java News and Posts. Get latest updates and posts on Java from Buggybread.com
Enter your email address:
Delivered by FeedBurner
 Sample 35. Clone a repository / checkout project in Git

git clone <repository_url>

   Like      Feedback     


 Sample 36. Location of .bash_profile in ubuntu

Its in your Home directory

   Like      Feedback     


 Sample 37. How to reload .bash_profile in ubuntu

source ~/.bash_profile

   Like      Feedback     


 Sample 38. Install ruby on ubuntu

sudo apt-get install ruby-full

   Like      Feedback     


 Sample 39. Install AWS CLI ( Amazon Web Service Command Line Interface ) using brew

brew install awscli

   Like      Feedback     


Subscribe to Java News and Posts. Get latest updates and posts on Java from Buggybread.com
Enter your email address:
Delivered by FeedBurner
 Sample 40. Install MySQL using brew

brew install mysql

   Like      Feedback     


 Sample 41. Install redis using brew

brew install redis

   Like      Feedback     


 Sample 42. Install AWS CLI on Linux / ubuntu

sudo apt-get install awscli

   Like      Feedback     


 Sample 43. Execute class with main method using maven

Execute it from the folder with pom file

mvn exec:java -Dexec.mainClass="com.*.*"

   Like      Feedback     


 Sample 44. Run df command ( check disk space usage ) using AWS CLI on AWS EC2 instance

Usage -

aws ssm send-command --document-name "AWS-RunPowerShellScript" --parameters commands=["df -a"] --targets "Key=instanceids,Values=<instanceId>"

Example -

aws ssm send-command --document-name "AWS-RunPowerShellScript" --parameters commands=["df -a"] --targets "Key=instanceids,Values= i-0981d8456ffd18wcb"

   Like      Feedback     aws cli  aws ssm  aws ssm send-command  check disk space usage on AWS instanc


Subscribe to Java News and Posts. Get latest updates and posts on Java from Buggybread.com
Enter your email address:
Delivered by FeedBurner
 Sample 45. Get a list of files within AWS S3 bucket

aws s3 ls s3://<bucketName>

Example -

aws s3 ls s3://myBucket/bucket

   Like      Feedback     aws s3  list files within aws s3


 Sample 46. Get Summary / List of files recursively from an S3 bucket

aws s3 ls s3://mybucket --recursive --human-readable --summarize

   Like      Feedback     amazon aws s3


 Sample 47. Load Data from CSV into MySql Table

USAGE

LOAD DATA LOCAL INFILE '<CSV_FILE_NAME>' INTO TABLE <TABLE_NAME> FIELDS TERMINATED BY '<FIELD_SEPARATOR>';

EXAMPLE

LOAD DATA LOCAL INFILE 'xyz.csv' INTO TABLE EMPLOYEE FIELDS TERMINATED BY ',';

   Like      Feedback     load csv into mysql table  csv to mysql


 Sample 48. Download S3 public file

wget <S3 URL>

   Like      Feedback     Download s3 file  s3 file  amazon s3


 Sample 49. Writes a single data record into an Amazon Kinesis stream

aws kinesis put-record --stream-name <STREAM_NAME> --partition-key <PARTITION_KEY> --data <DATA>

   Like      Feedback     Amazon AWS  AWS  Amazon Kinesis Stream  aws kinesis put-record  aws kinesis   put single record in amazon Kinesis


Subscribe to Java News and Posts. Get latest updates and posts on Java from Buggybread.com
Enter your email address:
Delivered by FeedBurner
 Sample 50. Get current Date in a specified format ( year, month,date,hour and minute) within shell script.

currentDate=$(date "+%Y-%m-%dT%H:%M")

   Like      Feedback     current date   current date in specific format


 Sample 51. Get information about a particular replication group

aws describe-replication-groups --replication-group-id <REPLICATION_GROUP_ID> --output text

   Like      Feedback     AWS  Amazon AWS  AWS CLI  describe-replication-groups


 Sample 52. Get Elastic Beanstalk / EC2 instance information using instance name

aws ec2 describe-instances --output text --filters "Name=tag:Name,Values=<ELASTIC_BEANSTALK_INSTANCE_NAME>"

   Like      Feedback     AWS  Amazon AWS  AWS CLI  describe-instances  Amazon EC2


 Sample 53. Get all Tag information for EC2 instance using instance name

aws ec2 describe-instances --output text --filters "Name=tag:Name,Values=<ELASTIC_BEANSTALK_INSTANCE_NAME>" | grep "TAGS"

   Like      Feedback     AWS  Amazon AWS  AWS CLI  describe-instances  Amazon EC2


 Sample 54. Get Private IP address of a EC2 / Beanstalk instance using AWS CLI

aws ec2 describe-instances --output text --filters "Name=tag:Name,Values=<ELASTIC_BEANSTALK_INSTANCE_NAME>" | grep "PRIVATEIPADDRESSES"

   Like      Feedback     AWS  Amazon AWS  AWS CLI  describe-instances  Amazon EC2


Subscribe to Java News and Posts. Get latest updates and posts on Java from Buggybread.com
Enter your email address:
Delivered by FeedBurner
 Sample 55. Remove all folders and subfolders recursively and forcibly

rm -rf *

   Like      Feedback     


 Sample 56. How to find lines in a file "File" containing "some text" ?

grep "some Text" File

   Like      Feedback     


 Sample 57. Command to list of jdk modules

java --list-modules

   Like      Feedback     java 9  java 9 module


 Sample 58. Get All attributes for a key

hgetall <KEY>

   Like      Feedback     Redis


 Sample 59. SSH into EC2 instance using pem ( key ) file

ssh -I <KEY_FILE_NAME> <USER_NAME>@<PUBLIC_IP>

   Like      Feedback     


Subscribe to Java News and Posts. Get latest updates and posts on Java from Buggybread.com
Enter your email address:
Delivered by FeedBurner
 Sample 60. Check if AWS is installed

which aws

   Like      Feedback     


 Sample 61. Get List of AWS Configuration

aws configure list

   Like      Feedback     


 Sample 62. Check if AWS has been configured properly

aws configure list | egrep "access_key|secret_key|region" | wc -l

Count of 3 means configured correctly

   Like      Feedback     


 Sample 63. Get Elastic Cache Information

aws elasticache describe-replication-groups --query 'ReplicationGroups[?ReplicationGroupId==`<ELASTIC_CACHE_IDENTIFIER>`].NodeGroups[0].{ip:PrimaryEndpoint.Address}' --output text

   Like      Feedback     


 Sample 64. Stop an EC2 instance through AWS CLI

aws ec2 stop-instances --instance-id=<EC2_INSTANCE_ID>

   Like      Feedback     


Subscribe to Java News and Posts. Get latest updates and posts on Java from Buggybread.com
Enter your email address:
Delivered by FeedBurner
 Sample 65. Start an EC2 instance using AWS CLI

aws ec2 start-instances --instance-id=<EC2_INSTANCE_ID>

   Like      Feedback     


 Sample 66. Instructing maven to set up files ( class path, project ) within eclipse

mvn eclipse:eclipse

   Like      Feedback     maven  eclipse


 Sample 67. Instructing maven to set up files ( class path, project ) within idea IntelliJ

mvn idea:idea

   Like      Feedback     maven  IntelliJ


 Sample 68. Set a Redis field value using key and field name

or Add a new Field to a Redis Key

HSET <Key> <field> <value>

   Like      Feedback     Redis  AWS


 Sample 69. Connect to Redis using Redis CLI

redis-cli -c -h <Redis End Point> -p <Port>

   Like      Feedback     Redis  aws


Subscribe to Java News and Posts. Get latest updates and posts on Java from Buggybread.com
Enter your email address:
Delivered by FeedBurner
 Sample 70. Run a program / command with security privileges of root / others

su

or

su <username>

or

sudo su <username>

   Like      Feedback     


 Sample 71. Build angular project

 ng build

   Like      Feedback     


 Sample 72. Running angular development server

ng serve

   Like      Feedback     


 Sample 73. Create mew angular component

ng generate component <component_name>

   Like      Feedback     


 Sample 74. Create new angular project using angular CLI

ng new <project_name>

   Like      Feedback     


Subscribe to Java News and Posts. Get latest updates and posts on Java from Buggybread.com
Enter your email address:
Delivered by FeedBurner
 Sample 75. Create new angular project using angular CLI with routing enabled

ng new <project_name> --routing

   Like      Feedback     


 Sample 76. Moving a file to staging

git add

   Like      Feedback     git


 Sample 77. Moving file from staging to head

git commit <file_name>

   Like      Feedback     git


 Sample 78. Moving file from Head on local to remote ?

git push <file_name>

   Like      Feedback     git


 Sample 79. Does git commit , commits the file to remote ?


   Like      Feedback     git


Subscribe to Java News and Posts. Get latest updates and posts on Java from Buggybread.com
Enter your email address:
Delivered by FeedBurner
 Sample 80. Setting the remote project for the Git initialized local project

git remote add origin <git_repo_url>

   Like      Feedback     git


 Sample 81. Merging remote changes to local

git pull

   Like      Feedback     git


 Sample 82. Add all files to the index for commit and push in git

git add .

   Like      Feedback     


 Sample 83. Steps in Git push from local to remote

git add .
git commit -m "message"
git push

   Like      Feedback     


 Sample 84. Commands and Steps for resolving Merge conflict in Git

git fetch ( it will only bring the remote code to local but won't attempt to merge )
git status ( will show the merge conflicts )
Fix the Merge conflicts
git add .
git commit
git push

   Like      Feedback     


Subscribe to Java News and Posts. Get latest updates and posts on Java from Buggybread.com
Enter your email address:
Delivered by FeedBurner
 Sample 85. Installing angular Redux module

npm install redux @angular-redux/store

   Like      Feedback     angular  redux


 Sample 86. Put a folder recursively using sftp

sftp <user_name>@<machine_address> 

put - r <folder_path>

   Like      Feedback     sftp



Subscribe to Java News and Posts. Get latest updates and posts on Java from Buggybread.com
Enter your email address:
Delivered by FeedBurner



comments powered by Disqus