This is more of a scratchpad and reference for me on various things I've found useful at somepoint and wanted to organize it all in one place. Most are just composed of pieces copied and modified a little from where I've found them.
- Cat a File with Line Numbers
- Change Default Shell for a User
- Change to that New Directory
- Check if a Port is in Use
- Check the Current Working Directory
- Clear the Screen
- Command Line Length Limitations
- Copying File Contents to System Paste Buffer
- Create a File Descriptor with Process Substitution
- Curling For Headers
- Curling with Basic Auth Credentials
- Display Free Disk Space
- Do Not Overwrite Existing Files
- Exclude a Directory with Find
- File Type Info with File
- Find Newer Files
- Get the Unix Timestamp
- Global Substitution on the Previous Command
- Grep for Files without a Match
- Grep for Multiple Patterns
- Hexdump a Compiled File
- Jump to the Ends of Your Shell History
- Kill Everything Running on a Certain Port
- Killing a Frozen SSH Session
- Last Argument of the Last Command
- List All Users
- List Names of Files with Matches
- List of Sessions to a Machine
- List Parent pid with ps
- Map a Domain to localhost
- Only Show the Matches
- Open the Current Command in an Editor
- Partial String Matching in Bash Scripts
- Repeat Yourself
- Saying Yes
- Search History
- Search man Page Descriptions
- Securely Remove Files
- Show Disk Usage for the Current Directory
- Sort in Numerical Order
- SSH Escape Sequences
- SSH with a Specific Key
- SSH with Port Forwarding
- Undo Some Command Line Editing
- View a Web Page in the Terminal
- Watch the Difference
- Watch This Run Repeatedly
- Where are the Binaries?
Cat a File with Line Numbers
You can quickly view a file using
cat but with the
-n flag you can view that file with line numbers
cat -n Gemfile 1 source 'https://rubygems.org' 2 3 4 # Bundle edge Rails instead: gem 'rails', github: 'rails/rails' 5 gem 'rails', '4.2.0' 6 # Use postgresql as the database for Active Record 7 gem 'pg'
Change Default Shell for a User
You can change the default shell program for a particular unix user with the
chsh command. Just tell it what shell program you want to use (e.g.
zsh) and which user the change is for:
[sudo] chsh -s /usr/bin/zsh username
This command needs to be invoked with root privileges. This command updates the entry for that user in the
Change to that New Directory
$_ variable provided by bash is always set to the last argument of the previous command. One handy use of this is for changing directories into a newly created directory. This command will leave you in your newly created directory,
mkdir new_dir && cd $_
Check if a Port is in Use
lsof command is used to list open files. This includes listing network connections. This means I can check if a particular port is in use and what process is using that port. For instance, I can check if a node.js application is currently running on port 3000.
$ lsof -i TCP:3000 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME node 13821 jason 12u IPv6 0xdf2e9fd346cc12b5 0t0 TCP localhost:hbci (LISTEN) node 13821 jason 13u IPv4 0xdf2e9fd33ca74d65 0t0 TCP localhost:hbci (LISTEN)
I can see that a node process (my node.js app) is using port 3000. The PID and a number of other details are included.
See more details with
Check the Current Working Directory
pwd to display the absolute path of the current working directory.
man pwd for more details.
Clear the Screen
If you type
clear into your shell, the screen will be cleared. There is a handy keybinding though that will save you a few keystrokes. Just hit
ctrl-l to achieve the same effect.
Command Line Length Limitations
The other day I tried to run a
rm command on the contents of a directory with a LOT of files.
Instead of deleting the contents of the directory, the following message was displayed:
/bin/rm: cannot execute [Argument list too long]
Bash wanted to expand the entire command before executing it. It was too long. But what is too long?
It turns out that we can figure out the max length of commands with the following command:
Copying File Contents to System Paste Buffer
If you need to copy and paste the contents of a file, the
pbcopy command can be one of the best ways to accomplish this. Simply
cat the file and pipe that into
pbcopy to get the contents of the file into the system paste buffer.
cat some-file.txt | pbcopy
man pbcopy for more details.
Create a File Descriptor with Process Substitution
Process substitution can be used to create a file descriptor from the evaluation of a shell command. The syntax for process substitution is
LIST is one or more bash commands.
cat <(echo 'hello, world') hello, world
This is particularly useful for commands that expect files, such as diff:
diff <(echo 'hello, world') <(echo 'hello, mars') 1c1 < hello, world --- > hello, mars
Curling For Headers
If you want to inspect the headers of a response from some endpoint, look no further than a quick
curl command. By including the
curl will return just the headers.
For example, if you are developing a web app that is being locally served at
localhost:3000 and you'd like to see what the headers look like for a particular URL, you might try something like the following command:
curl -I localhost:3000/posts
Curling with Basic Auth Credentials
I often use
curl to take a quick look at the responses of particular endpoints. If I try to
curl a URL that is secured with HTTP Basic Authentication, this is what the response looks like:
curl staging.example.com HTTP Basic: Access denied.
I can give the credentials to
curl so that it can plug them in as it makes the request using the
curl -u username:password staging.example.com <html><body>...</body></html>
If I don't want the password showing up in my command-line history, I can just provide the username and
curl will prompt me for my password:
curl -u username staging.example.com Enter host password for user 'username': <html><body>...</body></html>
man curl for more details.
Display Free Disk Space
df utility is a handy way to display the free disk space available on on a specific file system or all mounted file systems.
df with the
-h flag to display the disk space usage and availability in a human-readable format. Here is the output from a linode box of mine:
df -h Filesystem Size Used Avail Use% Mounted on /dev/xvda 20G 3.8G 16G 20% / none 4.0K 0 4.0K 0% /sys/fs/cgroup devtmpfs 994M 4.0K 994M 1% /dev none 200M 196K 199M 1% /run none 5.0M 0 5.0M 0% /run/lock none 996M 0 996M 0% /run/shm none 100M 0 100M 0% /run/user
Do Not Overwrite Existing Files
When using the
cp command to copy files, you can use the
-n flag to make sure that you do not overwrite existing files.
Exclude a Directory with Find
find is a handy way to track down files that meet certain criteria. However, if there are directories full of irrelevant files, you may end up with a lot of noise. What you want to do is exclude or ignore such directories. For example, you probably don't want
find to return results from the
.git directory of your project.
Specific directories can be excluded by combining the
For instance, to see all files modified within the last 10 days, but not including anything in the
.git directory, run the following:
find . -type f -not -path './.git/*' -ctime -10
File Type Info with File
file utility to determine the type of a file:
file todo.md todo.md: ASCII English text file Hello.java Hello.java: ASCII C++ program text file Hello.class Hello.class: compiled Java class data, version 52.0
Hello.java file isn't exactly a C++ program, but close enough.
Find Newer Files
-newer flag with the name of a file to find files that have a newer modification date than the named file. For instance,
find blog -name '*.md' -newer blog/first-post.md
will find all markdown files in the
blog directory that have a modification date more recent than
Get the Unix Timestamp
To get the Unix timestamp from your shell, use the
time command with the appropriate format:
Global Substitution on the Previous Command
Let's say we just executed the following command:
grep 'foo' foo.md
It gave us the information we were looking for and now we want to execute a similar command to find the occurrences of
^ trick won't quite work here.
^foo^bar<tab> grep 'bar' foo.md
What we need is a global replace of
foo in our previous command. The
!! command can help when we sprinkle in some
!!gs/foo/bar<tab> grep 'bar' bar.md
For a short command like this, we haven't gained much. However, for large commands that span the length of the terminal, this can definitely save us a little trouble.
Grep for Files without a Match
grep command is generally used to find files whose contents match a pattern. With the
grep can be used to find files that don't match the given pattern.
For instance, to find files in the current directory that don't have
foobar anywhere in their content, run:
grep -L "foobar" ./*
Grep for Multiple Patterns
You can use the
-e flag with the
grep command to search for a pattern. Additionally, you can use multiple
-e flags to search for multiple patterns. For instance, if you want to search for occurrences of
clojure in a
README.md file, use the following command:
grep -e ruby -e clojure README.md
man grep for more details.
Hexdump a Compiled File
hexdump unix utility allows you to dump the contents of a compiled/executable file in a readable hexadecimal format. Adding the
-C flag includes a sidebar with a formatted version of that row of hexadecimal.
For example, a compiled Hello World java program,
Hello.java, will look something like this:
cat Hello.class | hexdump -C 00000000 ca fe ba be 00 00 00 34 00 1d 0a 00 06 00 0f 09 |.......4........| 00000010 00 10 00 11 08 00 12 0a 00 13 00 14 07 00 15 07 |................| 00000020 00 16 01 00 06 3c 69 6e 69 74 3e 01 00 03 28 29 |.....<init>...()| 00000030 56 01 00 04 43 6f 64 65 01 00 0f 4c 69 6e 65 4e |V...Code...LineN| 00000040 75 6d 62 65 72 54 61 62 6c 65 01 00 04 6d 61 69 |umberTable...mai| 00000050 6e 01 00 16 28 5b 4c 6a 61 76 61 2f 6c 61 6e 67 |n...([Ljava/lang| 00000060 2f 53 74 72 69 6e 67 3b 29 56 01 00 0a 53 6f 75 |/String;)V...Sou| 00000070 72 63 65 46 69 6c 65 01 00 0a 48 65 6c 6c 6f 2e |rceFile...Hello.| 00000080 6a 61 76 61 0c 00 07 00 08 07 00 17 0c 00 18 00 |java............| 00000090 19 01 00 0d 48 65 6c 6c 6f 2c 20 57 6f 72 6c 64 |....Hello, World| 000000a0 21 07 00 1a 0c 00 1b 00 1c 01 00 05 48 65 6c 6c |!...........Hell| 000000b0 6f 01 00 10 6a 61 76 61 2f 6c 61 6e 67 2f 4f 62 |o...java/lang/Ob| 000000c0 6a 65 63 74 01 00 10 6a 61 76 61 2f 6c 61 6e 67 |ject...java/lang| 000000d0 2f 53 79 73 74 65 6d 01 00 03 6f 75 74 01 00 15 |/System...out...| 000000e0 4c 6a 61 76 61 2f 69 6f 2f 50 72 69 6e 74 53 74 |Ljava/io/PrintSt| 000000f0 72 65 61 6d 3b 01 00 13 6a 61 76 61 2f 69 6f 2f |ream;...java/io/| 00000100 50 72 69 6e 74 53 74 72 65 61 6d 01 00 07 70 72 |PrintStream...pr| 00000110 69 6e 74 6c 6e 01 00 15 28 4c 6a 61 76 61 2f 6c |intln...(Ljava/l| 00000120 61 6e 67 2f 53 74 72 69 6e 67 3b 29 56 00 20 00 |ang/String;)V. .| 00000130 05 00 06 00 00 00 00 00 02 00 00 00 07 00 08 00 |................| 00000140 01 00 09 00 00 00 1d 00 01 00 01 00 00 00 05 2a |...............*| 00000150 b7 00 01 b1 00 00 00 01 00 0a 00 00 00 06 00 01 |................| 00000160 00 00 00 01 00 09 00 0b 00 0c 00 01 00 09 00 00 |................| 00000170 00 25 00 02 00 01 00 00 00 09 b2 00 02 12 03 b6 |.%..............| 00000180 00 04 b1 00 00 00 01 00 0a 00 00 00 0a 00 02 00 |................| 00000190 00 00 03 00 08 00 04 00 01 00 0d 00 00 00 02 00 |................| 000001a0 0e |.| 000001a1
Jump to the Ends of Your Shell History
There are all sorts of ways to do things in your shell environment without reaching for the arrow keys. For instance, if you want to move up to the previous command, you can hit
Ctrl-p. To move down to the next command in your shell history, you can hit
But what if you want to move to the beginning and end of your entire shell history?
Find your meta key (probably the one labeled
alt) and hit
META-> to move to the end and beginning of your shell history, respectively.
Kill Everything Running on a Certain Port
You can quickly kill everything running on a certain port with the following command.
sudo kill `sudo lsof -t -i:3000`
This gets a list of pids for all the processes and then kills them.
Killing a Frozen SSH Session
Whenever an SSH session freezes, I usually mash the keyboard in desperation and then kill the terminal session. This can be avoided though. SSH will listen for the following kill command:
This will kill the frozen SSH session and leave you in the terminal where you were before you SSH'd.
Last Argument of the Last Command
You can use
!$ as a way to reference the last argument in the last command. This makes for an easy shortcut when you want to switch out commands for the same long file name. For instance, if you just ran
cat on a file to see its contents
and now you want to edit that file. You can just pass
!$ to the
Hit enter or tab to get the full command:
List All Users
On unix-based systems, all system users are listed with other relevant information in the
/etc/passwd file. You can output a quick list of the users by name with the following command:
cut -d: -f1 /etc/passwd
List Names of Files with Matches
I often use
ag to search for patterns in a group or directory of files. Generally I am interested in looking at the matching lines themselves. However, sometimes I just want to know the set of files that have matches. Both
ag can be told to output nothing more than the names of the files with matches when given the
This can come in particularly handy if you just want a list of files that can be piped (or copied) for use with another command. This eliminates all the extra noise.
List of Sessions to a Machine
last command is a handy way to find out who has been connecting to a machine and when.
Last will list the sessions of specified users, ttys, and hosts, in reverse time order. Each line of output contains the user name, the tty from which the session was conducted, any hostname, the start and stop times for the session, and the duration of the session. If the session is still continuing or was cut short by a crash or shutdown, last will so indicate.
In particular, this can be useful for finding an IP address that you want to connect to.
man last for more details.
List Parent pid with ps
ps command, which stands for
process status, is a great way to find different processes running on a machine. Information like their
pid (process id) is included. If you are tracking down a process to kill and find that that process is an unkillable zombie, then you may need to simultaneously kill the process' parent as well.
So, you'll need the parent
pid as well. You can get both the
pid and the parent
pid of a process by including the
-f flag with
You may also want to include the
-e flag to make sure that information about other users' processes is included in the results.
Map a Domain to localhost
Do you want your computer to treat a domain as
localhost? You can map it as such in your
/etc/hosts file. For example, if I have an web app that refers to itself with the
dev.app.com domain, I can add the following line to my
/etc/hosts file to make sure the domain resolves to
Now, if I pop open my browser and visit
dev.app.com:3000, I will see whatever is being served to
Only Show the Matches
ag make it easy to search for lines in a file that contain certain text and patterns. They all come with the
-o flag which tells them to only show the part that matches.
This is particularly powerful when used with regex and piped into other programs.
Open the Current Command in an Editor
If you are working with a complicated command in the terminal trying to get the arguments just right. Such as this
curl https://api.stripe.com/v1/customers \ -u sk_test_BQokikJOvBiI2HlWgH4olfQ2: \ -d description="Customer for email@example.com" \ -d source=tok_189fCz2eZvKYlo2CsGERUNIW
It can be tedious to move to and modify various parts of the command. However, by hitting
Ctrl-x Ctrl-e, the contents of the command buffer will be opened into your default editor (i.e.
$EDITOR). This will make editing the command a bit easier. Saving and quitting the editor will put the updated command in the command buffer, ready to run.
Ctrl-x Ctrl-e with an empty command buffer if you want to start crafting a command from scratch or if you are pasting one in from somewhere.
Partial String Matching in Bash Scripts
To compare two strings in a bash script, you will have a snippet of code similar to the following:
if [[ $(pwd) == "/path/to/current/directory" ]] then echo "You are in that directory"; fi
You may only want to do a partial string match. For this, you can use the
* wildcard symbol.
if [[ $(pwd) == *"directory"* ]] then echo "You are in that directory"; fi
repeat command to repeat some other command. You can repeat a command any number of times like so:
repeat 5 say Hello World
Tired of being prompted for confirmation by command-line utilities? Wish you could blindly respond 'yes' to whatever it is they are bugging you about? The
yes command is what you've been looking for.
yes | rm -r ~/some/dir
This will respond
rm asks for confirmation on removing each and every file in that directory.
yes is just as good at saying no. Give it
no as an argument and it will happily (and endlessly) print
Often times there is a very specific command you have entered into your bash prompt that you need to run again. You don't want to have to type it againand stepping manually through your history may be suboptimal if you typed it quite a while ago. Fortunately, there is a simple history search feature that you can use in this kind of situation.
Ctrl+r and then start typing a moderately specific search term. Your search history will be filtered by that term. Subsequent hitting of
Ctrl+r will step forward through that filtered history. Once you find the command you are looking for, hit enter to execute it.
Search man Page Descriptions
You can use the
apropos command with a keyword argument to search for that words occurrence throughout all the man pages on your system. For instance, invoking
apropos whatis will bring up a list of entries including the
apropos command itself.
man apropos for more details.
Securely Remove Files
If you really want to make sure you have wiped a file from your hard drive, you are going to want to use
srm instead of
rm. The man page for
srm gives the following description:
srm removes each specified file by overwriting, renaming, and truncating it before unlinking. This prevents other people from undeleting or recovering any information about the file from the command line.
Show Disk Usage for the Current Directory
du utility can be used to show disk usage for a particular directory or set of directories. When used without any arguments, it will show the disk usage for the current directory.
du 80 ./.git/hooks 8 ./.git/info 256 ./.git/logs/refs/heads ...
-h command we can see it all in a human-readable format
du -h 40K ./.git/hooks 4.0K ./.git/info 128K ./.git/logs/refs/heads
and to get an even clearer picture we can pipe that through
du -h | sort -nr 412K ./vim 352K ./postgres 340K ./.git/logs 216K ./.git/logs/refs 184K ./ruby 156K ./unix 148K ./git ...
This sorts it numerically in reverse order putting the largest stuff at the top.
Sort in Numerical Order
By default, the
sort command will sort things alphabetically. If you have numerical input though, you may want a numerical sort. This is what the
-n flag is for.
If I have a directory of files with numbered names, sort doesn't quite do the job by itself.
ls | sort 1.txt 10.txt 11.txt 12.txt 2.txt 3.txt 4.txt 5.txt
-n flag, I get the sort order I am looking for.
ls | sort -n 1.txt 2.txt 3.txt 4.txt 5.txt 10.txt 11.txt 12.txt
SSH Escape Sequences
In a previous post, I talked about an escape sequence for breaking out of an SSH session when the pipe has been broken. This isn't the only SSH escape sequence though. To see the others, hit
<Enter>~?. This displays a help list with all the other escape sequences.
~? Supported escape sequences: ~. - terminate connection (and any multiplexed sessions) ~B - send a BREAK to the remote system ~C - open a command line ~R - request rekey ~V/v - decrease/increase verbosity (LogLevel) ~^Z - suspend ssh ~# - list forwarded connections ~& - background ssh (when waiting for connections to terminate) ~? - this message ~~ - send the escape character by typing it twice (Note that escapes are only recognized immediately after newline.)
SSH with a Specific Key
When you SSH into another machine using public key authentication, the key pair from either
~/.ssh/id_rsa is used by default. This is generally what you want. But what if the target server is expecting to identify you with a different SSH key pair?
-i option can be used with
ssh to specify a different identity file when the default isn't what you want.
SSH with Port Forwarding
-L flag with
ssh to forward a connection to a remote server
ssh someserver -L3000:localhost:3000
Undo Some Command Line Editing
When using some of the fancy command line editing shortcuts, such as
ctrl-u, you may end up erroneously changing or deleting part of the current command. Retyping it may be a pain or impossible if you've forgotten exactly what was changed. Fortunately, bash's command line editing has undo built in. Just hit
ctrl-_ a couple times to get back to where you want to be.
View a Web Page in the Terminal
One such program is
link. If you don't already have it, you can install it with something like
homebrew. Then run
links with any URL for a magnificent, ad-free experience:
Watch the Difference
watch command is a simple way to repeatedly run a particular command. I'll sometimes use it to monitor the response from some endpoint.
watch can make monitoring responses even easier when the
-d flag is employed.
This flag instructs
watch to highlight the parts of the output that are different from the previous run of the command.
So if I run
watch -d curl -LIs localhost:3000
I can easily see if the http status of the request changes.
Watch This Run Repeatedly
I usually reach for a quick bash for loop when I want to run a particular process a bunch of times in a row. The
watch command is another way to run a process repeatedly.
watch rspec spec/some/test.rb
The default is 2 seconds in between subsequent executions of the command. The period can be changed with the
-n flag though:
watch -n 2 rspec spec/some/test.rb
Where are the Binaries?
When I want to know where an executable is, I use
which like so:
which rails /Users/jbranchaud/.gem/ruby/2.1.4/bin/rails
That is the rails binary on my path that will be used if I enter a rails command. However, with something like rails, there may be multiple versions on your path. If you want to know where all of them are, you can use
where rails /Users/jbranchaud/.gem/ruby/2.1.4/bin/rails /Users/jbranchaud/.rubies/2.1.4/bin/rails /usr/bin/rails