One day at a time

Here is the code to output result to csv file.

SELECT order_id,product_name,qty
FROM orders
INTO OUTFILE '/tmp/orders.csv'
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'

This is something related to sendmail IPv6 support.

1. cd /etc/mail
2. sendmail.mc and add the following at the end of the file:
3. define(`confBIND_OPTS’, `WorkAroundBrokenAAAA’)dnl
4. type make
5. restart sendmail

Here is the steps by steps hbase ec2 installation.

1. start elasticfox
   1. create a “keypairs”
   2. on your “security groups” default group, add port 50000-51000 (for hadoop) and port 60000-61000 (for hbase)
   3. search for hbase AMI (ami-4de30724)
   4. launch the install with the newly create keypair and default group permission
2. on you desktop linux/window machine do: ssh -i <keypair.pem> root@<your new instance IP>
   1. replace the <keypair.pem> with the newly create keypairs pem file
   2. replace <your new instance IP> to your newly create instance IP address
3. step 2  will login your ec2 machine. Then do the following:
   1. type: ssh-keygen -t dsa -P ” -f ~/.ssh/id_dsa
   2. type: cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
   3. type: ssh localhost
   4. If it does not ask you for the password, you are okay.
   5. type: exit
   6. vi /usr/local/hadoop-0.17.1/conf/hadoop-site.xml
      1. replace: <value>hdfs://:50001</value>  to <value>hdfs://INSTANCE IP ADDRESS:50001</value>
      2. replace: <value>:50002</value> to <value>INSTANCE IP ADDRESS:50002</value>
   7. type: /usr/local/hadoop-0.17.1/bin/hadoop dfs namenode -format
   8. type: /usr/local/hadoop-0.17.1/bin/start-all.sh
   9. check /mnt/hadoop/logs/hadoop-root-namenode-<….>.log and make sure that your hadoop is starting correctly.
   10. vi /usr/local/hbase-0.2.0/conf/hbase-site.xml
   11. replace:  <value>:60000</value> to  <value>INSTANCE IP ADDRESS:60000</value>
   12. replace:  <value>hdfs://:50001/hbase</value> to <value>hdfs://INSTANCE IP ADDRESS:50001/hbase</value>
   13. type: /usr/local/hbase-0.2.0/bin/start-hbase.sh
   14. check /mnt/hbase/logs/* to make sure your hbase is start correctly.
4. if everything is okay, you can point your browser to: http://INSTANCE IP ADDRESS:60010/ and see the hbase information.

Today, try to do a “yum update”.

It complains about:

Transaction Check Error:
file /etc/avahi/avahi-autoipd.action from install of avahi-autoipd-0.6.22-10.f
c9.i386 conflicts with file from package avahi-0.6.17-1.fc7.i386
file /usr/sbin/avahi-autoipd from install of avahi-autoipd-0.6.22-10.fc9.i386
conflicts with file from package avahi-0.6.17-1.fc7.i386
file /usr/share/man/man8/avahi-autoipd.8.gz from install of avahi-autoipd-0.6.
22-10.fc9.i386 conflicts with file from package avahi-0.6.17-1.fc7.i386

To fix it, do the following:

su
rpm -e –noscripts avahi-0.6.17-1.fc7.i386
yum update

Try to set up apache server with virtual hosts. When I restart the apache server, it keeps saying that:

_default_ VirtualHost overlap on port 80, the first has precedence

The problem is that the NameVirutalHost is commented. Make sure your NameVirtualHost line in httpd.conf is uncommented as:

NameVirtualHost *:80

I am trying to write a spider to scraoe Google result. Google returns different content for differnet user agent. Here is how to setup the user agent in curl.

$url=’http://www.google.com/search?hl=en&q=’.$keyword.’&start=10′;
$ch = curl_init($url);
$useragent=”Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1″;
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);

1. Logged in as root, check which timezone your machine is currently using by executing `date`. You’ll see something like Mon 17 Jan 2005 12:15:08 PM PST, PST in this case is the current timezone.
2. Change to the directory /usr/share/zoneinfo here you will find a list of time zone regions. Choose the most appropriate region, if you live in Canada or the US this directory is the “America” directory.
3. If you wish, backup the previous timezone configuration by copying it to a different location. Such as

   mv /etc/localtime  /etc/localtime-old

4. Create a symbolic link from the appropiate timezone to /etc/localtime. Example:

   ln -sf /usr/share/zoneinfo/Europe/Amsterdam /etc/localtime

5. If you have the utility rdate, update the current system time by executing

   /usr/bin/rdate -s time.nist.gov

6. Set the ZONE entry in the file /etc/sysconfig/clock file (e.g. “America/Los_Angeles”)
7. Set the hardware clock by executing:

   /sbin/hwclock --systohc

Try to setup Apache virtual host on a new machine.

 Apache keeps complain about about the DocumentRoot does not exist and cannot write the log file. My document is using capistrano to do symbol link.

 It turns out the problem is SELinux.

Do:

echo 0 > selinux/enforce

It will disable SELinux and fix the problem. I have not look into why yet… May be I should.

If you cannot do “gem install mysql”, it returns:

/usr/local/bin/ruby extconf.rb install mysql
checking for mysql_query() in -lmysqlclient… no
checking for main() in -lm… yes
checking for mysql_query() in -lmysqlclient… no
checking for main() in -lz… yes
checking for mysql_query() in -lmysqlclient… no
checking for main() in -lsocket… no
checking for mysql_query() in -lmysqlclient… no
checking for main() in -lnsl… yes
checking for mysql_query() in -lmysqlclient… no
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers. Check the mkmf.log file for more
details. You may need configuration options.

Try this:

gem install mysql — \
> –with-mysql-include=/usr/include/mysql \
> –with-mysql-lib=/usr/lib/mysql