Unix Material

1  1.  Introduction:

An operating system is the main software component of computer system. It provides users with an environment that makes it possible to use the hardware devices of computer. Examples of operating systems are Android, BSD, iOS, LINUX, Microsoft Windows, MAC OS X, and z/OS. Apart from Microsoft Windows and z/OS, all the other operating systems in the list are UNIX-based Operating System. An operating System creates an interface between user and the hardware.

The first version of UNIX was created in 1969 by Kenneth Thompson and Dennis Ritchie, system engineers at AT&T's Bell Labs. It went through many revisions and gained in popularity until 1977, when it was first made commercially available by Interactive Systems Corporation. At the same time a team from the University of California at Berkeley was working to improve UNIX. In 1977 it released the first Berkeley Software Distribution, which became known as BSD. Meanwhile the AT&T version was developing in different ways. The 1978 release of Version 7 included the Bourne Shell for the first time. By 1983 commercial interest was growing and Sun Microsystems produced a UNIX workstation.

1.1 Definition:

The Unix operating system is a set of programs that act as a link between the computer and the user. The computer programs that allocate the systemresources and coordinate all the details of the computer's internals is called the operating systemor the kernel.

1.2 Features of Unix Operating System:       

The following are the advantages of Unix Features.

Portability:                                                        

The system is written in high-level language making it easier to read, understand, change and, therefore move to other machines. The code can be changed and complied on a new machine. Customers can then choose from a wide variety of hardware vendors without being locked in with a particular vendor. 

Multi-Tasking:

Unix is a powerful multi-tasking operating system; it means when a active task in in process, there can be a simultaneous background process working too. Unix handles these active and background threads efficiently and manages the system resources in a fair-share manner.

Multi-User Operations: 

Unix is a multi-user system designed to support a group of users simultaneously. The system allows for the sharing of processing power and peripheral resources, white at the same time providing excellent security features. 

Hierarchical File System: 

Unix uses a hierarchal file structure to store information. This structure has the maximum flexibility in grouping information in a way that reflects its natural state. It allows for easy maintenance and efficient implementation. 

Unix shell: 

Unix has a simple user interface called the shell that has the power to provide the services that the user wants. It protects the user from having to know the intricate hardware details. 

Utilities: 

Unix has over 200 utility programs for various functions. New utilities can be built effortlessly by combining existing utilities. 

1.3 Structure of Unix Operating System:

          The structure of the Unix Operating System consists of four parts. They are hardware, kernel, shell and tools and applications.

Hardware: Hardware refers to the physical components that collectively form a computer machine.

The following are three primary components.

1.     I/O devices

2.     CPU

3.     Memory.

                                         

                                      Structure of Unix System

Kernel: The Kernel is the heart of the UNIX OS. It is a software application that provides the interface between the hardware and the user. It handles the process, memory, file and device and network management for the operating system. The kernel is responsible for ensuring that all system and user tasks are performed concurrently.

Shell: The shell is the program that sits between the user and the kernel. It is the interpreter that translates the commands that are typed into the terminal session. Users can type commands directly into the terminal, or they can create a text file containing a series of commands that can be sent to the shell. The series of commands are called a shell script.There are multiple shells that are used by the UNIX OS. They include the Bourne shell (sh), the C shell (csh), the Korn shell (ksh) and the Bourne Again shell (bash). Each shell has own set of shell commands. Operating system commands are the same across all the shells.

Tools and Applications: The final layer of the UNIX OS is the Utilities and Applications layer. This layer includes the commands, word processors, graphic programs and database management programs. Traditionally, these programs were accessed by typing the commands to start the program on the command line. They can still be accessed in this way, but they can now also be accessed through the GUI.

2. Unix File System:

2.1: Introduction to Files: A file is a container of text, images, codes, and  so on. Everything nis a file on Unix System. Files are ordered in a hierarchal tree-like fashion with root represented by the character ‘/’. The directories are the internal nodes of the tree structure, while the files are considered to be the leaves.

Types of Files:

The files are divides into following three categories in the Unix operating system.

i.                   Ordinary Files: These files contain only data.

ii.                 Directory Files: These files act as a container and can contain ordinary files and device files along with directory files.

iii.              Device files: These files represent all the hardware devices.

2.2: Organization of File Systems: Unix file system is a logical method of organizing and storing large amounts of information in a way that makes it easy to manage. A file is a smallest unit in which the information is stored. Unix file system has several important features. All data in Unix is organized into files. All files are organized into directories. These directories are organized into a tree-like structure called the file system.

Files in Unix System are organized into multi-level hierarchy structure known as a directory tree. At the very top of the file system is a directory called “root” which is represented by a “/”. All other files are “descendants” of root.

                                               

Directories or Files and their description:

/ : The slash / character alone denotes the root of the file system tree.

/bin : Stands for “binaries” and contains certain fundamental utilities, such as ls or cp, which are generally needed by all users.

/boot : Contains all the files that are required for successful booting process.

/dev : Stands for “devices”. Contains file representations of peripheral devices and pseudo-devices.

/etc : Contains system-wide configuration files and system databases. Originally also contained “dangerous maintenance utilities” such as init, but these have typically been moved to /sbin or elsewhere.

/home : Contains the home directories for the users.

/lib : Contains system libraries, and some critical files such as kernel modules or device drivers.

/media : Default mount point for removable devices, such as USB sticks, media players, etc.

/mnt : Stands for “mount”. Contains file system mount points. These are used, for example, if the system uses multiple hard disks or hard disk partitions. It is also often used for remote (network) file systems, CD-ROM/DVD drives, and so on.

/proc : procs virtual file system showing information about processes as files.

/root: The home directory for the super user “root” – that is, the system administrator. This account’s home directory is usually on the initial file system, and hence not in /home (which may be a mount point for another file system) in case specific maintenance needs to be performed, during which other file systems are not available. Such a case could occur, for example, if a hard disk drive suffers physical failures and cannot be properly mounted.

/tmp : A place for temporary files. Many systems clear this directory upon start up; it might have tmpfs mounted atop it, in which case its contents do not survive a reboot, or it might be explicitly cleared by a startup script at boot time. 

/usr : Originally the directory holding user home directories, its use has changed. It now holds executable, libraries, and shared resources that are not system critical, like the X Window System, KDE, Perl, etc. However, on some Unix systems, some user accounts may still have a home directory that is a direct subdirectory of /usr, such as the default as in Minx. (on modern systems, these user accounts are often related to server or system use, and not directly used by a person).

/var : A short for “variable.” A place for files that may change often – especially in size, for example e-mail sent to users on the system, or process-ID lock files.

2.3 Accessing File System: Before you can access the files on a file system, you need to mount the file system. Mounting a file system attaches that file system to a directory (mount point) and makes it available to the system. The root (/) file system is always mounted. Any other file system can be connected or disconnected from the root (/) file system.

When you mount a file system, any files or directories in the underlying mount point directory are unavailable as long as the file system is mounted. These files are not permanently affected by the mounting process, and they become available again when the file system is unmounted. However, mount directories are typically empty, because you usually do not want to obscure existing files.

For example, the figure below shows a local file system, starting with a root (/) file system and subdirectories sbin, etc, and opts.

                                                            Sample root (/) File System

                                          

Now, you wanted to access a local file system from the /opt file system that contains a set of unbundled products. First, you must create a directory to use as a mount point for the file system you want to mount, for example, /opt/unbundled. Once the mount point is created, you can mount the file system (by using the mount command), which makes all of the files and directories in /opt/unbundled available, as shown in the figure below.

                                                     Mounting a File System

                

Unmounting the file system means detaching the mounted file system from the directory of the Unix system on which it was mounted. Once, a file system is un mounted ,we will not be able to access its files or directories

2.4 Structure of File System:  Each physical drive can be divided into several partitions. Each partition can contain one file system. Each file system contains:

1.  boot block(s);

2.  superblock;

3.  inode list;

4.  data blocks.

 A boot block may contain the bootstrap code that is read into the machine upon booting.

A superblock describes the state of the file system:

• how large it is;
• how many files it can store;
• where to find free space on the file system;
• who has ownership of it;
• and more.

            The inode list is an array of "information nodes" analogous to the FAT (File Allocation Table) system in MS-DOS.

 Data blocks start at the end of the inode list and contain file data and directory blocks.

3.Unix Commands:

The UNIX operating system has for many years formed the backbone of the Internet, especially for large servers and most major university campuses. However, a free version of UNIX called Linux has been making significant gains against Macintosh and the Microsoft Windows 95/98/NT environments, so often associated with personal computers. Developed by a number of volunteers on the Internet such as the Linux group and the GNU project, much of the open-source software is copyrighted, but available for free. This is especially valuable for those in educational environments where budgets are often limited.

UNIX commands can often be grouped together to make even more powerful commands with capabilities known as I/O redirection ( < for getting input from a file input and > for outputing to a file ) and piping using | to feed the output of one command as input to the next. Please investigate manuals in the lab for more examples than the few offered here.

The following charts offer a summary of some simple UNIX commands. These are certainly not all of the commands available in this robust operating system, but these will help you get started.

Ten ESSENTIAL UNIX Commands:

These are ten commands that you really need to know in order to get started with UNIX. They are probably similar to commands you already know for another operating system.

Command	Example	Description
1.     ls	ls ls -a	Lists files in current directory List in long format
2.     cd	cd tempdir cd ..	Change directory to tempdir Move back one directory
3.     mkdir	mkdir graphics	Make a directory called graphics
4.     rmdir	rmdir emptydir	Remove directory (must be empty)
5.     cp	cp file1 web-docs cp file1 file1.bak	Copy file into directory Make backup of file1
6.     rm	rm file1.bak rm *.tmp	Remove or delete file Remove all file
7.     mv	mv old.html new.html	Move or rename files
8.     more	more index.html	Look at file, one page at a time
9.     lpr	lpr index.html	Send file to printer
10.   man	man ls	Online manual (help) about command

Ten VALUABLE UNIX Commands:

Once you have mastered the basic UNIX commands, these will be quite valuable in managing your own account.

Command	Example	Description
1.     grep <str><files>	grep "bad word" *	Find which files contain a certain word
2.     chmod <opt> <file>	chmod 644 *.html chmod 755 file.exe	Change file permissions read only Change file permissions to executable
3.     passwd	passwd	Change passwd
4.     ps <opt>	ps aux ps aux   |   grep dhyatt	List all running processes by #ID List process #ID's running by dhyatt
5.     kill <opt>    <ID>	kill -9 8453	Kill process with ID #8453
6.     gcc (g++) <source>	gcc file.c -o file g++ fil2.cpp -o fil2	Compile a program written in C Compile a program written in C++
7.     gzip <file>	gzip bigfile gunzip bigfile.gz	Compress file Uncompress file
8.     mail         (pine)	mail me@tjhsst.edu < file1 pine	Send file1 by email to someone Read mail using pine
9.     telnet <host>         ssh <host>	telnet	Open a connection
10.   ftp <host> ncftp <host/directory>	ftp station1.tjhsst.edu ncftp metalab.unc.edu	Upload or Download files to station1 Connect to archives at UNC

Ten FUN UNIX Commands

These are ten commands that you might find interesting or amusing. They are actually quite helpful at times, and should not be considered idle entertainment.

Command	Example	Description
1.     who	who	Lists who is logged on your machine
2.     finger	finger	Lists who is on computers in the lab
3.     ytalk <user@place>	ytalk dhyatt@threat	Talk online with dhyatt who is on threat
4.     history	history	Lists commands you've done recently
5.     fortune	fortune	Print random humerous message
6.     date	date	Print out current date
7.     cal <mo> <yr>	cal 9 2000	Print calendar for September 2000
8.     xeyes	xeyes &	Keep track of cursor (in "background")
9.     xcalc	xcalc &	Calculator ("background" process)
10.   mpage <opt> <file>	mpage -8 file1   |  lpr	Print 8 pages on a single sheet and send to printer (the font will be small!)

Ten HELPFUL UNIX Commands:

These ten commands are very helpful, especially with graphics and word processing type applications.

Command	Example	Description
1.     netscape	netscape &	Run Netscape browser
2.     xv	xv &	Run graphics file converter
3.     xfig / xpaint	xfig & (xpaint &)	Run drawing program
4.     gimp	gimp &	Run photoshop type program
5.     ispell <fname>	ispell file1	Spell check file1
6.     latex <fname>	latex file.tex	Run LaTeX, a scientific document tool
7.     xemacs / pico	xemacs (or pico)	Different editors
8.     soffice	soffice &	Run StarOffice, a full word processor
9.     m-tools (mdir, mcopy,         mdel, mformat, etc. )	mdir a: mcopy file1   a:	DOS commands from UNIX (dir A:) Copy file1 to A:
10.   gnuplot	gnuplot	Plot data graphically

The TEN Useful Commands in Unix:                                   

These ten commands are useful for monitoring system access, or simplifying your own environment.

Command	Example	Description
1.     df	df	See how much free disk space
2.     du	du -b subdir	Estimate disk usage of directory in Bytes
3.     alias	alias lls="ls -alF"	Create new command "lls" for long format of ls
4.     xhost	xhost + threat.tjhsst.edu xhost -	Permit window to display from x-window program from threat Allow no x-window access from other systems
5.     fold	fold -s file1   |   lpr	Fold or break long lines at 60 characters and send to printer
6.     tar	tar -cf subdir.tar subdir tar -xvf subdir.tar	Create an archive called subdir.tar of a directory Extract files from an archive file
7.     ghostview (gv)	gv filename.ps	View a Postscript file
8.     ping    (traceroute)	ping threat.tjhsst.edu traceroute www.yahoo.com	See if machine is alive Print data path to a machine
9.     top	top	Print system usage and top resource hogs
10.   logout (exit)	logout or exit	How to quit a UNIX shell.

                    

4.     About vi-Editor:

There are many ways to edit files in Unix. Editing files using the screen-oriented text editor vi is one of the best ways. This editor enables you to edit lines in context with other lines in the file.An improved version of the vi editor which is called the VIM has also been made available now. Here, VIM stands for Vi IMproved.

vi is generally considered the de facto standard in Unix editors because −

·         It's usually available on all the flavours of Unix system.

·         Its implementations are very similar across the board.

·         It requires very few resources.

·         It is more user-friendly than other editors such as the ed or the ex.

You can use the vi editor to edit an existing file or to create a new file from scratch. You can also use this editor to just read a text file.

Starting the vi Editor     

The following table lists out the basic commands to use the vi editor –

Sr.No.	Command & Description
1	vi filename            Creates a new file if it already does not exist, otherwise opens an  existing file.
2	vi -R filename            Opens an existing file in the read-only mode.
3	view filename           Opens an existing file in the read-only mode.

Following is an example to create a new file testfile if it already does not exist in the current working directory −

Example: $vi testfile

 

Operation Modes:

While working with the vi editor, we usually come across the following two modes −

·         Command mode − This mode enables you to perform administrative tasks such as saving the files, executing the commands, moving the cursor, cutting (yanking) and pasting the lines or words, as well as finding and replacing. In this mode, whatever you type is interpreted as a command.

·         Insert mode − This mode enables you to insert text into the file. Everything that's typed in this mode is interpreted as input and placed in the file.

vi always starts in the command mode. To enter text, you must be in the insert mode for which simply type i. To come out of the insert mode, press the Esc key, which will take you back to the command mode.

Steps:

1.To enter in to vi editor.. vi file name

2.Press ‘i’ to enter into insert mode.

3.Type data into the file.

4.Press escape to come from insert mode.

5.The press Shift+: for command mode

6. After ‘:’ press wq for save and quit,or q without save and quitting out from editor.

7. Type ‘cc’ for run and execution

8.Type ./a.out for output

5.     Shell Programming:

   A shell script is a text file that contains a sequence of commands for a UNIX-based operating system. It's called a shell script because it combines into a "script" in a single file a sequence of commands that would otherwise have to be presented to the system from a keyboard one at a time.

Shell Types:

In UNIX there are two major types of shells:

1.     The Bourne shell. If you are using a Bourne-type shell, the default prompt is the $ character.

2.     The C shell. If you are using a C-type shell, the default prompt is the % character.

There are again various subcategories for Bourne Shell which are listed as follows:

·         Bourne shell ( sh)

·         Korn shell ( ksh)

·         Bourne Again shell ( bash)

·         POSIX shell ( sh)

The different C-type shells follow:

·         C shell ( csh)

·         TENEX/TOPS C shell ( tcsh)

About BOURNE Shell: Bourne shell was one of the major shells used in the early versions of the Unix operating system. It represented by the dollar ‘$’ symbol I the command line prompt. The BOURNE shell provides variables, flow control constructs, and functions to write user-friendly and interactive shell scripts.

          Features of Bourne Shell:

1.     Allows execution of commands and scripts.

2.     Provides a set of built-in commands and utilities.

3.     Enables execution of commands in the background.

4.     Provides input/output redirection, pipes and filters.

5.     Provides commands for loops and conditional branching

6.     Enables command substitution by using back quotes.

7.     Support pattern matching operators(?,*)

Basic Shell Commands:

1.     echo: Displaying messages and values.

Syntax: echo “message”.

Example: echo “GVP”

2.     expr: Evaluating expressions.

Syntax: expr `arg1 operator arg2 ..`

Example: expr ` $q  + $a`

3.     let: Assigning and evaluating expressions

Syntax: let variable=value/expression

Example:let x=15+20

echo $x

25

4.     bc: Base Conversion

The calculator mode is invoked by typing bc command at all the shell prompt.

Syntax: bc

Example: bc

10/2

2

Quit

On typing quit the bc commands ends.

5.     factor:  Factorizing Numbers

Syntax: factor [number]

Example: $ factor

                   15

                             3

                             5

                             q

It waits for another number when you didn’t quit. You want to quit from factor use zero or any non-numerical character.

6.     units: Scale Conversion

The units command converts quantities expressed in various standard scales to their equivalents in other scales.

Synatax and Example: units

                                      You have : inch

                                      You want: cm

                                      *2.540000+00

                                      Exit

7.     read: Reading input from users

The read command is used to rad the input typed by the user into shell variables.

Syntax: read  variable name

Example: read  a

8.     test/ [ ]:((test command): Testing expression for validity

Syntax and example: if test $a –ge $b (or) if [ $a –ge $b ]

About While Loop: The while loop is used for repeating a set of statements for the time the specified logical expression is true.
Syntax: while [ logical expression ]
                   do
                             …..
                   done
Example: while [ $n –le 10 ]
                   do
                             echo $n
                             let n++
                   done

About Until loop: The until loop is used for repeating a set of statements for the time the specified logical expression is false. The moment the logical expression becomes true, the control will come out of the loop.

Syntax: until [logical expression]

                   do

                             ….

                   done

Example: n=1

                 until test $n –gt 10

                   do

                             echo $n >> filename

                             ((n=n+1))

                   done

About if Statement: The if statement is used for selecting a set of statements out of the two sets depending on the validity of the logical expression included.

Syntax: if (logical expression) then

                   cmd1

                   …….

                   Else

                   Cmd2

                   …….

              fi

Example: m=60

                   if [$m –ge 60 ]

                   then

                   echo “first division”

                   else

                   echo “second division”

                   fi

6.AWK Command:

          AWK command in Unix/Linux with examples. Awk is a scripting language used for manipulating data and generating reports.The awk command programming language requires no compiling, and allows the user to use variables, numeric functions, string functions, and logical operators.

          Awk is a utility that enables a programmer to write tiny but effective programs in the form of statements that define text patterns that are to be searched for in each line of a document and the action that is to be taken when a match is found within a line. Awk is mostly used for pattern scanning and processing. It searches one or more files to see if they contain lines that matches with the specified patterns and then performs the associated actions.

Awk is abbreviated from the names of the developers – Aho, Weinberger, and Kernighan.

WHAT CAN WE DO WITH AWK ?

1.     AWK Operations:
(a) Scans a file line by line
(b) Splits each input line into fields
(c) Compares input line/fields to pattern
(d) Performs action(s) on matched lines

2.      Useful For:
(a) Transform data files
(b) Produce formatted reports

3.      Programming Constructs:
(a) Format output lines
(b) Arithmetic and string operations
(c) Conditionals and loops

Advantages and Disadvantages of using AWK Filters:
The following are the advantages of AWK filters:

1.     It uses very simple patterns and actions that are easy to learn.

2.     It is quite small in size

3.     It is an interpreted language, hence it consumes fewer resources.

4.     It accomplishes complex filtering tasks with minimum instructions

5.     It considers text files as records and fields.

6.     It applies the desired filtering and processing operations on the fields and generates formatted reports.

The following are the disadvantages of AWK filters:

1.     It processes the file sequentially, hence consumes much time in filtering.

2.     It is not suitable for filtering large volume of data.

AWK has features for filtering, text processing and writing reports. It operates at the field level and easily accesses transform and format individual fields in a record.

The syntax of AWK command is as follows:

  Syntax: awk ‘/pattern/’ {actions} file(s)