Posts

Showing posts with the label linux

Hostname and mail configurations in Linux

By default mail command will set the from address as $user@$hostname in Linux. If you want to change this behavior using the configuration file (/etc/mail.rc ) this is how it can be done. Set the entire from address like below. Add this to the configuration file. set from="from address" If you need to change the hostname part only, add this. set hostname="hostname"

Bind a remote server's port to a local port

If you have a remote server (say in Amazon EC2) you might want to access a particular port of that server. But there can be situations where it is not that port is not globally open. If you do not want to bother making it globally open you can use it by binding it to a local port of your workstation via ssh. Following command will bind port 9000 of remote machine to your local port 8000.  As an example if it is web server you can easily access it by typing localhost:8000 in your web browser. ssh -L 8000:localhost:9000 username@host

Optimizing Apache Storm deployment

Recently we happen to run some pretty large Storm topologies in a Storm cluster which runs on Linux. When we running it there were two main issues occurred due to system limitations. First one was logged in Storm logs as, “java.lang.OutOfMemoryError : unable to create new native Thread”. We fixed this problem by increasing the Ulimit  for Storm. Usually storm spawns processes with an user named Storm. So we have to increase the Ulimit for storm user. You can see the ulimits using command "ulimit -u". To increase the ulimits you can follow an approach like this . The second problem was communication link failures between Storm nodes as well as communication link failures between other services (e.g. external APIs, databases, etc). To resolve this problem we had to enable tcp time wait reuse and also we increased the port range for tcp. hat can be done in following manner.  Put these to /etc/sysctl.conf file and issue 'sysctl -f'     net.ipv4.tcp_tw_reus...