Tuesday, July 30, 2013

Critical Multipath parameters in Linux

# man multipath.conf

fast_io_fail_tmo
Specify the number of seconds the scsi layer will wait after a problem has been detected on a FC remote port before failing IO to devices on that remote port. This should be smaller than dev_loss_tmo. Setting this to off will disable the timeout.

dev_loss_tmo
Specify the number of seconds the scsi layer will wait after a problem has been detected on a FC remote port before removing it from the system. This can be set to "infinity" which sets it to the max value of 2147483647 seconds, or 68 years.
----------------

Regarding fail_if_no_path, please check:
----------------
https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/5/html/DM_Multipath/queueifnopath_issues.html

5.4. Issues with queue_if_no_path feature

If features "1 queue_if_no_path" is specified in the /etc/multipath.conf file, then any process that issues I/O will hang until one or more paths are restored. To avoid this, set the no_path_retry N parameter in the /etc/multipath.conf file (where N is the number of times the system should retry a path).

When you set the no_path_retry parameter, remove the features "1 queue_if_no_path" option from the /etc/multipath.conf file as well. If, however, you are using a multipathed device for which the features "1 queue_if_no_path" option is set as a compiled-in default, as it is for many SAN devices, you must explicitly add features "0" to override this default. You can do this by copying the existing devices section for your device from /usr/share/doc/device-mapper-multipath-0.4.7/multipath.conf.defaults into /etc/multipath.conf and editing it to suit your needs.

If you need to use the features "1 queue_if_no_path" option and you experience the issue noted here, use the dmsetup command to edit the policy at runtime for a particular LUN (that is, for which all the paths are unavailable). For example, if you want to change the policy on the multipath device mpath2 from "queue_if_no_path" to "fail_if_no_path", execute the following command.

dmsetup message mpath2 0 "fail_if_no_path"

Note that you must specify the mpathn alias rather than the path. 

Configuring UDEV in Linux

This section is for systems with external or networked storage; that is, Fibre Channel or iSCSI based storage devices. It is recommended that those systems have persistent device names configured for your hosts. This assists live migration as well as providing consistent device names and storage for multiple virtualized systems.

Universally Unique Identifiers(UUIDs) are a standardized method for identifying computers and devices in distributed computing environments. This sections uses UUIDs to identify iSCSI or Fibre Channel LUNs. UUIDs persist after restarts, disconnection and device swaps. The UUID is similar to a label on the device.

Systems which are not running multipath must use single path config. Systems running multipath can use multiple path config.

Single path configuration
This procedure implements LUN device persistence using udev. Only use this procedure for hosts which are not using multipath.

  1. Edit the /etc/scsi_id.config file.

    1. Ensure the options=-b is line commented out.
      # options=-b
      

    2. Add the following line:
      options=-g
      

      This option configures udev to assume all attached SCSI devices return a UUID.

  2. To display the UUID for a given device run the scsi_id -g -s /block/sd* command. For example:
    # scsi_id -g -s /block/sd*
    3600a0b800013275100000015427b625e
    

    The output may vary from the example above. The output displays the UUID of the device /dev/sdc.

  3. Verify the UUID output by the scsi_id -g -s /block/sd* command is identical from computer which accesses the device.

  4. Create a rule to name the device. Create a file named 20-names.rules in the /etc/udev/rules.d directory. Add new rules to this file. All rules are added to the same file using the same format. Rules follow this format:
    KERNEL=="sd[a-z]", BUS=="scsi", PROGRAM="/sbin/scsi_id -g -s /block/%k", RESULT="UUID", NAME="devicename"
    

    Replace UUID and devicename with the UUID retrieved above, and a name for the device. This is a rule for the example above:
    KERNEL="sd*", BUS="scsi", PROGRAM="/sbin/scsi_id -g -s", RESULT="3600a0b800013275100000015427b625e", NAME="rack4row16"
    

    The udev daemon now searches all devices named /dev/sd* for the UUID in the rule. Once a matching device is connected to the system the device is assigned the name from the rule. In the a device with a UUID of 3600a0b800013275100000015427b625e would appear as /dev/rack4row16.

  5. Append this line to /etc/rc.local:
    /sbin/start_udev
    

  6. Copy the changes in the /etc/scsi_id.config, /etc/udev/rules.d/20-names.rules, and /etc/rc.local files to all relevant hosts.
    /sbin/start_udev
    

Networked storage devices with configured rules now have persistent names on all hosts where the files were updated This means you can migrate guests between hosts using the shared storage and the guests can access the storage devices in their configuration files.

Multiple path configuration
The multipath package is used for systems with more than one physical path from the computer to storage devices. multipath provides fault tolerance, fail-over and enhanced performance for network storage devices attached to Red Hat Enterprise Linux systems.

Implementing LUN persistence in a multipath environment requires defined alias names for your multipath devices. Each storage device has a UUID which acts as a key for the aliased names. Identify a device's UUID using the scsi_id command.
# scsi_id -g -s /block/sdc

The multipath devices will be created in the /dev/mpath directory. In the example below 4 devices are defined in /etc/multipath.conf:
multipaths { 
 multipath { 
 wwid  3600805f30015987000000000768a0019 
 alias  oramp1 
 } 
 multipath { 
 wwid  3600805f30015987000000000d643001a 
 alias  oramp2 
 } 
 mulitpath { 
 wwid  3600805f3001598700000000086fc001b 
 alias  oramp3 
 } 
 mulitpath { 
 wwid  3600805f300159870000000000984001c 
 alias  oramp4 
 } 
}

This configuration will create 4 LUNs named /dev/mpath/oramp1, /dev/mpath/oramp2, /dev/mpath/oramp3 and /dev/mpath/oramp4. Once entered, the mapping of the devices' WWID to their new names are now persistent after rebooting.   

Out of Memory Error

If your server hangs / crashes after throwing errors in messages such as follows :-

Jul 18 03:07:00 xxxx kernel:  [<ffffffff810a170a>] ? cpuset_print_task_mems_allowed+0x92/0x9e
Jul 18 03:07:00 xxxx kernel:  [<ffffffff810da742>] oom_kill_process+0x85/0x25b
Jul 18 03:07:00 xxxx kernel:  [<ffffffff810dac18>] ? select_bad_process+0xbc/0x102
Jul 18 03:07:00 xxxx kernel:  [<ffffffff810dac9b>] __out_of_memory+0x3d/0x86
Jul 18 03:07:00 xxxx kernel:  [<ffffffff810daf6b>] out_of_memory+0xfc/0x195

======

Try and find out the following

1. Check memory usage at the time system triggered OOM , check how much is free memory

cache + buffer + free / total memory = If this value is less then 20% or so , your system is definitely under memory pressure.

2. Check if system is swapping or not , this can be checked at cat /proc/meminfo and the swapping data should look like this :-

SwapTotal:      12578852 kB
SwapFree:        4765960 kB


which means system has been swapping, and ideally system should never swap and only under memory pressure does system swap.

3. Check the value of AnonPages in cat /proc/meminfo, basically this value indicates the memory which has been assigned to processes but forgotten to be taken back from then. (malloc) , if this value is significant percentage of your total memory, then you got the root cause of the issue.
================
There are few kernel tweaks which could be done to ease out memory pressure, keep

vm.min_free_kbytes = 512000
vm.swappiness = 100 
vm.zone_reclaim_mode = 1

in /etc/sysctl.conf  and then reload the file via the command sysctl -p . If it is a database server, try and configure hugepages for your setup. get help of google to setup hugepages.

Tuesday, January 22, 2013

Repairing Corrupted GRUB in Linux

1. Insert the dvd(or 1st cd ) of RHEL5.
2. when your linux cd boot your pc and you have a prompt like this 

boot:_

Write linux rescue command here

boot: linux rescue 

3. Now select the language and other appropriate options

4. Then select continue to mount your partition in read and write mode.

5. After mounting your partition in read and write mode type the command given below when you have sh shell prompt 

chroot /mnt/sysimage 

6. Now type this command 

grub-install /dev/hda (if you have sata hard disk then use sda in place of hda)

7. Type exit 

8. Type exit again 

9. System will Reboot !!! enjoy

Friday, January 18, 2013

Disabling a CPU in running Linux System


  • The following command will disable a CPU:
# echo 0 > /sys/devices/system/cpu/cpuX/online
where X is the ID of the CPU as determined from /proc/cpuinfo.
  • To re-enable the CPU, run:
# echo 1 > /sys/devices/system/cpu/cpuX/online
verify  by # cat /proc/cpuinfo 

Thursday, June 21, 2012

ulimit in Linux Explained.


ulimit

User limits - limit the use of system-wide resources.
Syntax
      ulimit [-acdfHlmnpsStuv] [limit]

Options

   -S   Change and report the soft limit associated with a resource. 
   -H   Change and report the hard limit associated with a resource. 

   -a   All current limits are reported. 
   -c   The maximum size of core files created. 
   -d   The maximum size of a process's data segment. 
   -f   The maximum size of files created by the shell(default option) 
   -l   The maximum size that may be locked into memory. 
   -m   The maximum resident set size. 
   -n   The maximum number of open file descriptors. 
   -p   The pipe buffer size. 
   -s   The maximum stack size. 
   -t   The maximum amount of cpu time in seconds. 
   -u   The maximum number of processes available to a single user. 
   -v   The maximum amount of virtual memory available to the process. 
ulimit provides control over the resources available to the shell and to processes started by it, on systems that allow such control.
The soft limit is the value that the kernel enforces for the corresponding resource. The hard limit acts as a ceiling for the soft limit.
An unprivileged process may only set its soft limit to a value in the range from 0 up to the hard limit, and (irreversibly) lower its hard limit. A privileged process may make arbitrary changes to either limit value.
If limit is given, it is the new value of the specified resource. Otherwise, the current value of the soft limit for the specified resource is printed, unless the `-H' option is supplied.
When setting new limits, if neither `-H' nor `-S' is supplied, both the hard and soft limits are set.
Restricting per user processes ( -u) can be useful for limiting the potential effects of a fork bomb.

Values are in 1024-byte increments, except for `-t', which is in seconds, `-p', which is in units of 512-byte blocks, and `-n' and `-u', which are unscaled values.
The return status is zero unless an invalid option is supplied, a non-numeric argument other than unlimited is supplied as a limit, or an error occurs while setting a new limit.
ulimit is a bash built in command.


ulimit example
# ulimit -a 
core file size          (blocks, -c) 0 
data seg size           (kbytes, -d) unlimited 
file size               (blocks, -f) unlimited 
pending signals                 (-i) 8191 
max locked memory       (kbytes, -l) 32 
max memory size         (kbytes, -m) unlimited 
open files                      (-n) 1024 
pipe size            (512 bytes, -p) 8 
POSIX message queues     (bytes, -q) 819200 
stack size              (kbytes, -s) 8192 
cpu time               (seconds, -t) unlimited 
max user processes              (-u) 8191 
virtual memory          (kbytes, -v) unlimited 
file locks                      (-x) unlimited 


Creating Network Bond in Linux


Linux network Bonding is creation of a single bonded interface by combining 2 or more Ethernet interfaces. This helps in high availability of your network interface and offers performance improvement. Bonding is same as port trunking or teaming.

Bonding allows you to aggregate multiple ports into a single group, effectively combining the bandwidth into a single connection. Bonding also allows you to create multi-gigabit pipes to transport traffic through the highest traffic areas of your network. For example, you can aggregate three megabits ports into a three-megabits trunk port. That is equivalent with having one interface with three megabytes speed

Steps for bonding in Oracle Enterprise Linux and Redhat Enterprise Linux are as follows..

Step 1.

Create the file ifcfg-bond0 with the IP address, netmask and gateway. Shown below is my test bonding config file.

$ cat /etc/sysconfig/network-scripts/ifcfg-bond0

DEVICE=bond0
IPADDR=192.168.1.12
NETMASK=255.255.255.0
GATEWAY=192.168.1.1
USERCTL=no
BOOTPROTO=none
ONBOOT=yes
Step 2.

Modify eth0, eth1 and eth2 configuration as shown below. Comment out, or remove the ip address, netmask, gateway and hardware address from each one of these files, since settings should only come from the ifcfg-bond0 file above. Make sure you add the MASTER and SLAVE configuration in these files.

$ cat /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0
BOOTPROTO=none
ONBOOT=yes
# Settings for Bond
MASTER=bond0
SLAVE=yes

$ cat /etc/sysconfig/network-scripts/ifcfg-eth1

DEVICE=eth1
BOOTPROTO=none
ONBOOT=yes
USERCTL=no
# Settings for bonding
MASTER=bond0
SLAVE=yes

$ cat /etc/sysconfig/network-scripts/ifcfg-eth2

DEVICE=eth2
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes

Step 3.

Set the parameters for bond0 bonding kernel module. Select the network bonding mode based on you need. The modes are

mode=0 (Balance Round Robin)
mode=1 (Active backup)
mode=2 (Balance XOR)
mode=3 (Broadcast)
mode=4 (802.3ad)
mode=5 (Balance TLB)
mode=6 (Balance ALB)
Add the following lines to /etc/modprobe.conf # bonding commands
alias bond0 bonding
options bond0 mode=1 miimon=100

Step 4.

Load the bond driver module from the command prompt.

$ modprobe bonding

Step 5.

Restart the network, or restart the computer.

$ service network restart # Or restart computer

When the machine boots up check the proc settings.

$ cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.0.2 (March 23, 2006)

Bonding Mode: adaptive load balancing
Primary Slave: None
Currently Active Slave: eth2
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth2
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:13:72:80: 62:f0

Look at ifconfig -a and check that your bond0 interface is active. You are done!.

To verify whether the failover bonding works..
Do an ifdown eth0 and check /proc/net/bonding/bond0 and check the “Current Active slave”.
Do a continuous ping to the bond0 ipaddress from a different machine and do a ifdown the active interface. The ping should not break.

Tuesday, June 5, 2012

RAID Levels Explained



RAID 0, RAID 1, RAID 5, RAID 10 Explained with Diagrams



RAID stands for Redundant Array of Inexpensive (Independent) Disks.


On most situations you will be using one of the following four levels of RAIDs.


§ RAID 0


§ RAID 1


§ RAID 5


§ RAID 10 (also known as RAID 1+0)





This article explains the main difference between these raid levels along with an easy to understand diagram.

In all the diagrams mentioned below:


§ A, B, C, D, E and F – represents blocks


§ p1, p2, and p3 – represents parity







RAID LEVEL 0



Following are the key points to remember for RAID level 0.



§ Minimum 2 disks.


§ Excellent performance ( as blocks are striped ).


§ No redundancy ( no mirror, no parity ).


§ Don’t use this for any critical system.




RAID LEVEL 1


Following are the key points to remember for RAID level 1.


§ Minimum 2 disks.


§ Good performance ( no striping. no parity ).


§ Excellent redundancy ( as blocks are mirrored ).





RAID LEVEL 5



Following are the key points to remember for RAID level 5.


§ Minimum 3 disks.


§ Good performance ( as blocks are striped ).


§ Good redundancy ( distributed parity ).


§ Best cost effective option providing both performance and redundancy. Use this for DB that is heavily read oriented. Write operations will be slow.







RAID LEVEL 10





Monday, June 4, 2012

Linux Bash Shell Options


Displaying options
Use the -o option to set to display all shell options:
willy:~> set -o
allexport              off
braceexpand            on
emacs                  on
errexit                off
hashall                on
histexpand             on
history                on
ignoreeof              off
interactive-comments   on
keyword                off
monitor                on
noclobber              off
noexec                 off
noglob                 off
nolog                  off
notify                 off
nounset                off
onecmd                 off
physical               off
posix                  off
privileged             off
verbose                off
vi                     off
xtrace                 off
See the Bash Info pages, section Shell Built-in Commands->The Set Built-in for a description of each option. A lot of options have one-character shorthands: the xtrace option, for instance, is equal to specifying set -x.
Changing options

Shell options can either be set different from the default upon calling the shell, or be set during shell operation. They may also be included in the shell resource configuration files.
The following command executes a script in POSIX-compatible mode:
willy:~/scripts> bash --posix script.sh
For changing the current environment temporarily, or for use in a script, we would rather use set. Use - (dash) for enabling an option, + for disabling:
willy:~/test> set -o noclobber

willy:~/test> touch test

willy:~/test> date > test
bash: test: cannot overwrite existing file

willy:~/test> set +o noclobber

willy:~/test> date > test
The above example demonstrates the noclobber option, which prevents existing files from being overwritten by redirection operations. The same goes for one-character options, for instance -u, which will treat unset variables as an error when set, and exits a non-interactive shell upon encountering such errors:
willy:~> echo $VAR


willy:~> set -u

willy:~> echo $VAR
bash: VAR: unbound variable
This option is also useful for detecting incorrect content assignment to variables: the same error will also occur, for instance, when assigning a character string to a variable that was declared explicitly as one holding only integer values.
One last example follows, demonstrating the noglob option, which prevents special characters from being expanded:
willy:~/testdir> set -o noglob

willy:~/testdir> touch *

willy:~/testdir> ls -l *
-rw-rw-r--    1 willy    willy        0 Feb 27 13:37 *

Linux Boot Sequence



6 Stages of Linux Boot Process (Startup Sequence)

Press the power button on your system, and after few moments you see the Linux login prompt.

Have you ever wondered what happens behind the scenes from the time you press the power button until the Linux login prompt appears?

The following are the 6 high level stages of a typical Linux boot process.


1. BIOS
  • BIOS stands for Basic Input/Output System
  • Performs some system integrity checks
  • Searches, loads, and executes the boot loader program.
  • It looks for boot loader in floppy, cd-rom, or hard drive. You can press a key (typically F12 of F2, but it depends on your system) during the BIOS startup to change the boot sequence.
  • Once the boot loader program is detected and loaded into the memory, BIOS gives the control to it.
  • So, in simple terms BIOS loads and executes the MBR boot loader.

2. MBR
  • MBR stands for Master Boot Record.
  • It is located in the 1st sector of the bootable disk. Typically /dev/hda, or /dev/sda
  • MBR is less than 512 bytes in size. This has three components 1) primary boot loader info in 1st 446 bytes 2) partition table info in next 64 bytes 3) mbr validation check in last 2 bytes.
  • It contains information about GRUB (or LILO in old systems).
  • So, in simple terms MBR loads and executes the GRUB boot loader.

3. GRUB
  • GRUB stands for Grand Unified Bootloader.
  • If you have multiple kernel images installed on your system, you can choose which one to be executed.
  • GRUB displays a splash screen, waits for few seconds, if you don’t enter anything, it loads the default kernel image as specified in the grub configuration file.
  • GRUB has the knowledge of the filesystem (the older Linux loader LILO didn’t understand filesystem).
  • Grub configuration file is /boot/grub/grub.conf (/etc/grub.conf is a link to this). The following is sample grub.conf of CentOS.

#boot=/dev/sda

default=0

timeout=5

splashimage=(hd0,0)/boot/grub/splash.xpm.gz

hiddenmenu

title CentOS (2.6.18-194.el5PAE)

          root (hd0,0)

          kernel /boot/vmlinuz-2.6.18-194.el5PAE ro root=LABEL=/

          initrd /boot/initrd-2.6.18-194.el5PAE.img
  • As you notice from the above info, it contains kernel and initrd image.
  • So, in simple terms GRUB just loads and executes Kernel and initrd images.

4. Kernel
  • Mounts the root file system as specified in the “root=” in grub.conf
  • Kernel executes the /sbin/init program
  • Since init was the 1st program to be executed by Linux Kernel, it has the process id (PID) of 1. Do a ‘ps -ef | grep init’ and check the pid.
  • initrd stands for Initial RAM Disk.
  • initrd is used by kernel as temporary root file system until kernel is booted and the real root file system is mounted. It also contains necessary drivers compiled inside, which helps it to access the hard drive partitions, and other hardware.

5. Init
  • Looks at the /etc/inittab file to decide the Linux run level.
  • Following are the available run levels
    • 0 – halt
    • 1 – Single user mode
    • 2 – Multiuser, without NFS
    • 3 – Full multiuser mode
    • 4 – unused
    • 5 – X11
    • 6 – reboot
  • Init identifies the default initlevel from /etc/inittab and uses that to load all appropriate program.
  • Execute ‘grep initdefault /etc/inittab’ on your system to identify the default run level
  • If you want to get into trouble, you can set the default run level to 0 or 6. Since you know what 0 and 6 means, probably you might not do that.
  • Typically you would set the default run level to either 3 or 5.

6. Runlevel programs
  • When the Linux system is booting up, you might see various services getting started. For example, it might say “starting sendmail …. OK”. Those are the runlevel programs, executed from the run level directory as defined by your run level.
  • Depending on your default init level setting, the system will execute the programs from one of the following directories.
    • Run level 0 – /etc/rc.d/rc0.d/
    • Run level 1 – /etc/rc.d/rc1.d/
    • Run level 2 – /etc/rc.d/rc2.d/
    • Run level 3 – /etc/rc.d/rc3.d/
    • Run level 4 – /etc/rc.d/rc4.d/
    • Run level 5 – /etc/rc.d/rc5.d/
    • Run level 6 – /etc/rc.d/rc6.d/
  • Please note that there are also symbolic links available for these directory under /etc directly. So, /etc/rc0.d is linked to /etc/rc.d/rc0.d.
  • Under the /etc/rc.d/rc*.d/ directories, you would see programs that start with S and K.
  • Programs starts with S are used during startup. S for startup.
  • Programs starts with K are used during shutdown. K for kill.
  • There are numbers right next to S and K in the program names. Those are the sequence number in which the programs should be started or killed.
  • For example, S12syslog is to start the syslog deamon, which has the sequence number of 12. S80sendmail is to start the sendmail daemon, which has the sequence number of 80. So, syslog program will be started before sendmail.

There you have it. That is what happens during the Linux boot process.

Sunday, May 27, 2012

Saturday, May 26, 2012

understanding Linux configuration files


Every Linux program is an executable file holding the list of opcodes the CPU executes to accomplish specific operations. For instance, the ls command is provided by the file /bin/ls, which holds the list of machine instructions needed to display the list of files in the current directory onto the screen. The behaviour of almost every program can be customized to your preferences or needs by modifying its configuration files.
In a word, no. Users who are new to Linux (rightly) feel frustrated that each configuration file looks like a new challenge to figure out. In Linux each programmer is free to choose the configuration file format he or she prefers. Format options range from the /etc/shells file, which contains a list of possible shells separated by a newline, to Apache's complex /etc/httpd.conf file.
The kernel itself may be considered a "program." Why does the kernel need configuration files? The kernel needs to know the list of users and groups in the system, and manage file permissions (that is, determine if a file can be opened by a specific user, according to the permissions, UNIX_USERS). Note that these files are not specifically read by programs, but by a function provided by a system library, and used by the kernel. For instance, a program needing the (encrypted) password of a user should not open the /etc/passwd file. Instead, it should call the system library function getpw(). This kind of function is also known as a system call. It is up to the kernel (through the system library) to open the /etc/passwd file and after that, search for the password of the requested user.
Most of the configuration files in the Red Hat Linux system are in the /etc directory unless otherwise specified. The configuration files can be broadly classified into the following categories:
/etc/host.confTells the network domain server how to look up hostnames. (Normally /etc/hosts, then name server; it can be changed through netconf.)
/etc/hostsContains a list of known hosts (in the local network). Can be used if the IP of the system is not dynamically generated. For simple hostname resolution (to dotted notation), /etc/hosts.conf normally tells the resolver to look here before asking the network nameserver, DNS or NIS.
/etc/hosts.allowMan page same as hosts_access. Read by tcpd at least.
/etc/hosts.denyMan page same as hosts_access. Read by tcpd at least.
/etc/issue & /etc/issue.netThese files are read by mingetty (and similar programs) to display a "welcome" string to the user connecting from a terminal (issue) or through a telnet session (issue.net). They include a few lines stating the Red Hat release number, name, and Kernel ID. They are used by rc.local.
/etc/redhat-releaseIncludes one line stating the Red Hat release number and name. Used by rc.local.
/etc/rc.d/rcNormally run for all run levels with level passed as argument. For example, to boot your machine in the Graphics mode (X-Server), run the following command from your command line: init 5. The runlevel 5 is starts the system in graphics mode.
/etc/rc.d/rc.localNot official. May be called from rc, rc.sysinit, or /etc/inittab.
/etc/rc.d/rc.sysinitNormally the first script run for all run levels.
/etc/rc.d/rc/rcX.dScripts run from rc (X stands for any number from 1 to 5). These directories are "run-level" specific directories. When a system starts up, it identifies the run-level to be initiated, and then it calls all the startup scripts present in the specific directory for that run-level. For example, the system usually starts up and the message "entering run-level 3" is shown after the boot messages; this means that all the init scripts in the directory /etc/rc.d/rc3.d/ will be called.
The kernel provides an interface to display some of its data structures that can be useful for determining the system parameters like interrupts used, devices initialised, memory statistics, etc. This interface is provided as a separate but dummy filesystem known as the /proc filesystem. Many system utilities use the values present in this filesystemf or displaying the system statistics. For example, the file /proc/modules lists the currently loaded modules in the system. This information is read by the command lsmod, which then displays it in a human readable format. In the same way, the file mtab specified in the following table reads the /proc/mount file, which contains the currently mounted filesystems.
/etc/mtabThis changes continuously as the file /proc/mount changes. In other words, when filesystems are mounted and unmounted, the change is immediately reflected in this file.
/etc/fstabLists the filesystems currently "mountable" by the computer. This is important because when the computer boots, it runs the command mount -a, which takes care of mounting every file system marked with a "1" in the next-to-last column of fstab.
/etc/mtools.confConfiguration for all the operations (mkdir, copy, format, etc.) on a DOS-type filesystem.
/etc/groupContains the valid group names and the users included in the specified groups. A single user can be present in more than one group if he performs multiple tasks. For example, is a "user" is the administrator as well as a member of the project group "project 1", then his entry in the group file will look like: user: * : group-id : project1
/etc/nologinIf the file /etc/nologin exists, login(1) will allow access only to root. Other users will be shown the contents of this file and their logins refused.
etc/passwdSee "man passwd". Holds some user account info including passwords (when not "shadowed").
/etc/rpmrcrpm command configuration. All the rpm command line options can be set together in this file so that all of the options apply globally when any rpm command is run on that system.
/etc/securettyContains the device names of tty lines (one per line, without leading /dev/) on which root is allowed to login.
/etc/usertty
/etc/shadow
Contains the encrypted password information for users' accounts and optionally the password aging information. Included fields are:
  • Login name
  • Encrypted password
  • Days since Jan 1, 1970 that password was last changed
  • Days before password may be changed
  • Days after which password must be changed
  • Days before password is to expire that user is warned
  • Days after password expires that account is disabled
  • Days since Jan 1, 1970 that account is disabled
/etc/shellsHolds the list of possible "shells" available to the system.
/etc/motdMessage Of The Day; used if an administrator wants to convey some message to all the users of a Linux server.
/etc/gated.confConfiguration for gated. Used only by the gated daemon.
/etc/gated.versionContains the version number of the gated daemon.
/etc/gatewayOptionally used by the routed daemon.
/etc/networksLists names and addresses of networks accessible from the network to which the machine is connected. Used by route command. Allows use of name for network.
/etc/protocolsLists the currently available protocols. See the NAG (Network Administrators Guide) and man page.
C interface is getprotoent. Should never change.
/etc/resolv.confTells the kernel which name server should be queried when a program asks to "resolve" an IP Address.
/etc/rpcContains instructions/rules for RPC, which can be used in NFS calls, remote file system mounting, etc.
/etc/exportsThe file system to be exported (NFS) and permissions for it.
/etc/servicesTranslates network service names to port number/protocol. Read by inetd, telnet, tcpdump, and some other programs. There are C access routines.
/etc/inetd.confConfig file for inetd. See the inetd man page. Holds an entry for each network service for which inetd must control daemons or other servicers. Note that services will be running, but comment them out in /etc/services so they will not be available even if running. Format:
/etc/sendmail.cfThe Mail program sendmail's configuration file. Cryptic to understand.
/etc/sysconfig/networkIndicates NETWORKING=yes or no. Read by rc.sysinit at least.
/etc/sysconfig/network-scripts/if*Red Hat network configuration scripts.
System commands are meant exclusively to control the system, and make everything work properly. All the programs like login (performing the authentication phase of a user on the console) or bash (providing the interaction between a user and the computer) are system commands. The files associated with them are therefore particularly important. This category has the following files of interest to users and administrators.
/etc/lilo.confContains the system's default boot command line parameters and also the different images to boot with. You can see this list by pressing Tab at the LILO prompt.
/etc/logrotate.confMaintains the log files present in the /var/log directory.
/etc/identd.confIdentd is a server that implements the TCP/IP proposed standard IDENT user identification protocol as specified in the RFC 1413 document. identd operates by looking up specific TCP/IP connections and returning the user name of the process owning the connection. It can optionally return other information instead of a user name. See the identd man page.
/etc/ld.so.confConfiguration for the Dynamic Linker.
/etc/inittabThis is chronologically the first configuration file in UNIX. The first program launched after a UNIX machine is switched on is init, which knows what to launch, thanks to inittab. It is read by init at run level changes, and controls the startup of the main process.
/etc/termcapA database containing all of the possible terminal types and their capabilities.
A daemon is a program running in non-interactive mode. Typically, daemon tasks are related to the networking area: they wait for connections, so that they can provide services through them. Many daemons are available for Linux, ranging from Web servers to ftp servers.
/etc/syslogd.confThe configuration file for the syslogd daemon. syslogd is the daemon that takes care of logging (writing to disk) messages coming from other programs to the system. This service, in particular, is used by daemons that would not otherwise have any means of signaling the presence of possible problems or sending messages to users.

/etc/httpd.conf
The configuration file for Apache, the Web server. This file is typically not in /etc. It may be in /usr/local/httpd/conf/ or /etc/httpd/conf/, but to make sure, you need to check the particular Apache installation.
/etc/conf.modules or /etc/modules.confThe configuration file for kerneld. Ironically, it is not the kernel "as a daemon". It is rather a daemon that takes care of loading additional kernel modules "on the fly" when needed.
In Linux (and UNIX in general), there are countless "user" programs. A most common user program config file is /etc/lynx.cfg. This is the configuration file for lynx, the well-known textual browser. Through this file you can define the proxy server, the character set to use, and so on. The following code sample shows a part of the lynx.cfg file that can be modified to change the proxy settings of the Linux system. These settings apply (by default) to all the users running lynx in their respective shells, unless a user overrides the default config file by specifying --cfg = "mylynx.cfg.

Proxy settings in /etc/lynx.cfg 
.h1 proxy
.h2 HTTP_PROXY
.h2 HTTPS_PROXY
.h2 FTP_PROXY
.h2 GOPHER_PROXY
.h2 NEWS_PROXY
.h2 NNTP_PROXY
# Lynx version 2.2 and beyond supports the use of proxy servers that can act as
# firewall gateways and caching servers. They are preferable to the older
# gateway servers. Each protocol used by Lynx can be mapped separately using
# PROTOCOL_proxy environment variables (see Lynx Users Guide). If you have 
# not set them externally, you can set them at run time via this configuration file.
# They will not override external settings. The no_proxy variable can be used
# to inhibit proxying to selected regions of the Web (see below). Note that on
# VMS these proxy variables are set as process logicals rather than symbols, to
# preserve lowercasing, and will outlive the Lynx image.
#
.ex 15
http_proxy:http://proxy3.in.ibm.com:80/
ftp_proxy:http://proxy3.in.ibm.com:80/
#http_proxy:http://penguin.in.ibm.com:8080
#ftp_proxy:http://penguin.in.ibm.com:8080/

.h2 NO_PROXY
# The no_proxy variable can be a comma-separated list of strings defining
# no-proxy zones in the DNS domain name space.  If a tail substring of the
# domain-path for a host matches one of these strings, transactions with that
# node will not be proxied.
.ex
no_proxy:demiurge.in.ibm.com, demiurge


When changing a configuration file, make sure that the program using that configuration is restarted if it's not controlled by the system administrator or the kernel. A normal user doesn't usually have privileges to start or stop system programs and/or daemons.
Changing configuration files in the kernel immediately affects the system. For example, changing the passwd file to add a user immediately enables that user. Also there are some kernel tunable parameters in the /proc/sys directory on any Linux system. The write-access to all these files is given only to the super-user; other users have only readonly access. The files in this directory are classified in the same manner as the Linux kernel source. Every file in this directory represents a kernel data structure that can be dynamically modified to change the system performance.
Note: Before changing any value in any of these files, make sure you know everything about the file to avoid irreparable damage to the system.
Files in the /proc/sys/kernel/ directory
File nameDescription
threads-maxThe maximum number of tasks the kernel can run.
ctrl-alt-delIf 1, then pressing this key sequence cleanly reboots the system.
sysrqIf 1, then Alt-SysRq is active.
osreleaseDisplays the release of the operating system.
ostypeDisplays the type of the operating system.
hostnameThe host name of the system.
domainnameNetwork domain of which the system is a part.
modprobeSpecifies whether modprobe should be automatically run at startup, and load the necessary modules.
A daemon is a program that is always running in background, quietly carrying out its task. Common ones are in.ftpd (ftp server daemon), in.telnetd (telnet server daemon), and syslogd (system logging daemon). Some daemons, while running, keep a close watch on the configuration file and reload it automatically when it changes. But most of the daemons do not reload automatically. We need to "tell" them somehow that the configuration file has changed and that it should be reloaded. This can be achieved (on Red Hat Linux systems) by restarting the services using the service command.
For example, if we have changed the network configuration, we need to issue:
service network restart.
Note: The services are most commonly the scripts present in the /etc/rc.d/init.d/* directory and are started by the init when the system is booted. So, to restart the service you can also do the following:
/etc/rc.d/init.d/ start | stop | status
start, stop, and status are the values that these scripts take as input to perform the action.
A user or system program reads its configuration file every time it is launched. Remember, though, that some system programs are spawned when the computer is turned on, and their behaviour depends on what they read in the configuration files in /etc/. So, the first time a user program is started, the default configuration is read from the files present in the /etc/ directory. Later, the user can customise the programs by using rc and . (dot) files as explained in the next section.
We have seen how programs can be easily configured. But what if someone does not like the way a program has been configured in /etc/? A "normal" user cannot simply go into /etc and change the configuration files; they are owned -- from the filesystem's point of view -- by root! This is why most user programs define two configuration files: the first one at a "system" level, located in /etc/; and the other one, "private" to the user, that can be found in his or her home directory.
For example, in my system I have installed the very useful wget utility. In /etc/ there is an /etc/wgetrc file. In my home directory, there is a file named .wgetrc, which describes my customised configuration (which will be loaded only when I, the user run the wget command). Other users may also have the .wgetrc file in their home directory (/home/other); this file will be read, of course, only when the user runs the wget command. In other words, the /etc/wgetrc file provides "default" values for wget, while the /home/xxx/.wgetrc file lists the "customisations" for a certain user. It is important to understand that this is the "general rule," and is not necessarily true for all cases. A program like pine, for instance, does not have any files in /etc/, but only the custom configuration in the users' home directory, in a file named .pinerc. Other programs may only have a default configuration file in /etc/, and may not let users "customize" them (it's the case with only a few of the config. files in the /etc dir.).

Commonly used rc and . (dot) files
FilenameDescription
~/.bash_loginLook at "man bash". Treated by bash like ~/.bash_profile if that doesn't exist.
~/.bash_logoutLook at "man bash".Sourced by bash login shells at exit.
~/.bash_profileSourced by bash login shells after /etc/profile.
~/.bash_historyThe list of commands executed previously.
~/.bashrcĂ‚ Look at "man bash". Sourced by bash non-login interactive shells (no other files are). Non-interactive shells source nothing unless BASH_ENV or ENV are set.
~/.emacsRead by emacs at startup.
~/.forward If this contains an e-mail address, then all mail to owner of ~ will be forwarded to that e-mail address.
~/.fvwmrc ~/.fvwm2rcConfig files for fvwm and fvwm2 (the basic X Window manager).
~/.hushloginLook at "man login". Causes a "quiet" login (no mail notice, last login info, or MOD).
~/.mail.rcUser init file for mail program.
~/.ncftp/Directory for ncftp program; contains bookmarks, log, macros, preferences, trace. See man ncftp. The purpose of ncftp is to provide a powerful and flexible interface to the Internet standard File Transfer Protocol. It is intended to replace the stock ftp program that comes with the system.
~/.profileLook at "man bash". Treated by bash like ~/.bash_profile if that and ~/.bash_login don't exist, and used by other Bourn-heritage shells too.
~/.pinercPine configuration
~/.muttrcMutt configuration
~/.exrcConfiguration of vi can be controlled by this file.
Example: set ai sm ruler
Writing the above line in this file makes vi set the auto-indentation, matching brackets and displaying line number and rows-columns options.
~/.vimrcDefault "Vim" configuration file. Same as .exrc.
~/.gtkrcGNOME Toolkit.
~/.kdercKDE configuration.
~/.netrcDefault login names and passwords for ftp.
~/.rhostsUsed by the r-tools: rsh, rlogin, etc. Very weak security since host impersonation is easy.
  1. Must be owned by user (owner of ~/) or superuser.
  2. Lists hosts from which users may access this account.
  3. Ignored if it is a symbolic link.
~/.rpmrcSee "man rpm". Read by rpm if /etc/rpmrc is not present.
~/.signatureMessage text that will be appended automatically to the mail sent from this account.
~/.twmrcConfig file for twm (The Window Manager).
~/.xinitrcRead by X at startup (not by xinit script). Mostly starts some progs.
Example: exec /usr/sbin/startkde
If the above line is present in this file, then the KDE Window Manager is started in when the startx command is issued from this account.
~/.xmodmaprcThis file is passed to the xmodmap program, and could be named anything (~/.Xmodmap and ~/.keymap.km, for example).
~/.xserverrcRun by xinit as the X server if it can find X to execute.
~/News/Sent-Message-IDsDefault mail history file for gnus.
~/.XauthorityRead and written by xdm program to handle authorization. See the X, xdm, and xauth man pages.
~/.Xdefaults,
~/.Xdefaults-hostname
Read by X applications during startup on hostname. If the -hostname file can't be found, .Xdefaults is looked for.
~/.XmodmapPoints to .xmodmaprc; Red Hat had (has) .xinitrc using this name.
~/.XresourcesUsually the name for the file passed to xrdb to load the X resources database, to avoid the need for applications to read a long .Xdefaults file. (~/.Xres has been used by some.)

~/mbox

User's old mail.