SethSec

Use dropbox to collect wedding photos (or any other group photos) with friends and family

2009-10-27T12:11:00.003-04:00

After my wedding, I searched for hours trying to find the easiest way for all my friends and family to send me their digital photos. An FTP server would have worked, but would have been slightly more intimidating to the non-technical folk. I decided that a shared dropbox account was the perfect solution, and it worked out even better than I could have imagined. I received 1900 unique pictures in 2 weeks!!

I created a new account, and sent the following email to everyone who attended:

Thanks so much for making our special day an amazing one! Now that it is over we would love to get everyone's perspective. So, we would appreciate it if you could drop your photos in our dropbox (instructions below).

This is an account that we will all share. This means that you can all check back in a week and download as many pictures as you like, and it also means that you can potentially delete all the pictures everyone else has uploaded… so be careful!

How to upload pictures:

Browse to this website: http://www.getdropbox.com/
Log in using this username/password combo (login is in upper right corner):
Username:
Password:
Once you get logged in click on the word Photos
If you would like us to know which pictures are yours, create a folder with your name
ie: Seth's Pics
Click on the folder you just created
Click the UPLOAD BUTTON

There are three ways to upload pictures:

Use the mass upload feature

This is by far the easiest.
Click the blue button that says "Choose Files"
Hold shift and can use the arrow keys to select up to 50 pictures at a time and select "open"
Then click submit to upload them to the site!

If that doesn’t work, you can try the “Basic Uploader”

This one is much slower because you have to select one file at a time like in most email clients. If the first option doesn’t work, I recommend going to the third.

Install the software package on your computer

Hit cancel, and then goto the “Install” link on the top right of the page.
Install the software package using the same username and password from above
This will create a “sync” folder on your computer where you can just copy all of the files from the original folder to this sync folder. Within minutes they will all upload to the server automatically!
*** This means that every picture someone else uploads will also automatically sync to your computer, so be mindful that this can eat up all of your bandwidth and/or hard drive space. ***

Thanks again!

Mysql fix

2008-06-11T20:55:00.005-04:00

I don't believe this post will be very useful to anyone else, but I want to record it anyway. I noticed a few weeks ago that my drupal installation was complaining that the watchdog table had crashed. With my limited understanding of mysql, I didn't event know that a table *could* crash. Everything else on the site looked fine to the anonymous user so I just ignored it.

That brings me to today. I found this interesting script online that will dump all of my mysql databases every hour to another file system. I figured I would give a shot. I entered my root db password and the dst directory and let her rip. I got a few errors right away:

[root@www storage]# mysql-backup.sh

mysqldump: Error 1194: Table 'watchdog' is marked as crashed and should be repaired when dumping table `watchdog` at row: 283

mysqldump: Got error: 145: Table './drupal/watchdog' is marked as crashed and should be repaired when using LOCK TABLES

mysqldump: Got error: 145: Table './gallery2/g2_CacheMap' is marked as crashed and should be repaired when using LOCK TABLES

Google quickly found the following post: http://gallery.menalto.com/node/72721, where a user kindly posted the solution to their own problem:

Stage 1: Checking your tablesRun myisamchk *.MYI or myisamchk -e *.MYI if you have more time. Use the -s (silent) option to suppress unnecessary information.If the mysqld server is stopped, you should use the --update-state option to tell myisamchk to mark the table as “checked.”You have to repair only those tables for which myisamchk announces an error. For such tables, proceed to Stage 2.If you get unexpected errors when checking (such as out of memory errors), or if myisamchk crashes, go to Stage 3.
Stage 2: Easy safe repairFirst, try myisamchk -r -q tbl_name (-r -q means “quick recovery mode”). This attempts to repair the index file without touching the data file. If the data file contains everything that it should and the delete links point at the correct locations within the data file, this should work, and the table is fixed.source: Website

[root@www storage]# myisamchk -r -q /var/lib/mysql/drupal/watchdog.MYI

- check record delete-chain

- recovering (with sort) MyISAM-table '/var/lib/mysql/drupal/watchdog.MYI'Data records: 513

- Fixing index 1

Found block that points outside data file at 122592

MyISAM-table '/var/lib/mysql/drupal/watchdog.MYI' is not fixed because of errors

Try fixing it by using the --safe-recover (-o), the --force (-f) option or by not using the --quick (-q) flag

[root@www storage]# myisamchk -r /var/lib/mysql/drupal/watchdog.MYI

- recovering (with sort)

MyISAM-table '/var/lib/mysql/drupal/watchdog.MYI'

Data records: 513

- Fixing index 1

Found block that points outside data file at 122592

- Fixing index 2

As you can see, I had to get rid of the -q option to get it to work, but it did in fact work. Same command worked to fix g2_CasheMap, but that one took quite a bit longer.

Looks like that did the trick.

Some snort login kung-fu...

2008-06-06T13:14:00.005-04:00

I was recently playing around with my .bash_profile file looking for new ways to alert myself as well as my team to problems with production snorts. I ended up with two little tricks that I have found really useful and I figured I would share.

For those that don't know, the .bash_profile file is an sh script that runs at user login. At a bare minimum it sets the users PATH, but it can be used for a whole lot more. It's located in the root of the users home directory. Ex: /home/snort/.bash_profile, or /root/.bash_profile

Before I go any further I will tell you that both of these tricks are obviously reactive in nature. They only let you know there is a problem the next time you log into the device. A more proactive solution would involve setting thresholds and sending emails to admins, but 1) there are already plenty of scripts that do that, and 2) that is not a luxury I have on my sensors. I have inbound ssh, outbound 80 for updates and outbound 443 for logging.

Nevertheless, this reactive approach is much cooler than nothing at all and I think it would still be helpful on any snort installation no matter what other active health monitoring is in place.

Display most recent snort signatures on login

The first function I added to .bash_profile is called checksnortsigs(). It sorts the files in /etc/snort/rules by date order, and grabs the date of the most recent .rules file. It's that simple. It then just prints that information when you log in and gives a little advice on what to do next:

checksnortsigs()
{
if [ -f /etc/init.d/snortd ]; then
LATESTRULE=`ls -ltr /etc/snort/rules/*.rules tail -1 awk '{print $6, $7}'`
echo "-------------- Snort Installation Detected -----------------"
echo "The most recent snort rules on this machine were updated on:"
echo " ******* $LATESTRULE *******"
echo "If the date above is more than 1 month old, run oinkmaster"
echo "manually and verify it completes without error."
echo "------------------------------------------------------------"
echo
fi
}

checksnortsigs

Output (which is displayed as soon as the user logs in) looks like this:

Last login: Thu May 29 16:27:36 2008 from xxxxxxxx
-------------- Snort Installation Detected -----------------
The most recent snort rules on this machine were updated on:
******* May 30 *******
If the date above is more than 1 month old, run oinkmaster
manually and verify it completes without error.
------------------------------------------------------------

Display % dropped packets and Mbps stats on login

Shortly after that, and this came to me after playing around with sguil and seeing how nicely the snort.stats information is integrated into the analyst console, I decided that I also wanted to display recent % dropped packets and Mbps statistics each time someone logged in. There are a few more steps to get this one working, but they are all very easy:

1) Enable the following line in snort.conf:

preprocessor perfmonitor: time 300 file /var/log/snort/snort.stats pktcnt 10000

2) Restart snort

3) I created a very simple bash script that is basically one line of code along with a bit of "usage" code to make it easier for others to run. I called it get-snort-stats.sh. I created it for the bash_profile script, but it can be used as a standalone program at all. Here it is:

#!/bin/bash
# A very simple utility that will display the % dropped packets as well as throughput statistics.
# Snort records this information every 5 minutes
# Author: Seth Art
# Created: May 20th, 2008

###########################
#Usage
###########################
if [ -z $1 ]; then
echo
echo "Usage: get-snort-stats [number of lines to display]..."
echo exit
fi

case $1 in
'-h''--help')
echo
echo "Usage: get-snort-stats [number of lines to display]..."
echo " -h, --help display this help and exit"
echo
exit 1
;;esac

###########################
#Main
###########################
tail -$1 /var/log/snort/snort.stats awk -F , '{print "Dropped Packets = " $2, "\t", "Mbps = "$3}'

4) This bit ties the get-snort-stats command in with the .bash_profile script. Add the following function to .bash_profile

getsnortstats()
{
if [ -f /etc/init.d/snortd ]; then
echo "------------------------------------------------------------"
echo" Snort % Pkts dropped and mbits/sec for the last 20 minutes "
/usr/local/bin/get-snort-stats.sh 4
echo "------------------------------------------------------------"
fi
}

5) Add the call to the getsnortstats() function right below the checksnortsigs() fucntion call in bash_profile:

checksnortsigs
getsnortstats

5) Now I'm positive there is a better way to do this, but to make sure the snort.stats file doesn't grow out of control I simply put a line that rm's snort.stats every night in the same script I ued to run oinkmaster, recreate sig-msg.map, and restart snort. Not the most elegant solution I know, but it works...

When all is said and done, you should see the following information when you log in from now on:

Last login: Thu May 29 16:27:36 2008 from xxxxxxxx
-------------- Snort Installation Detected -----------------
The most recent snort rules on this machine were updated on:
******* May 30 *******
If the date above is more than 1 month old, run oinkmaster
manually and verify it completes without error.
------------------------------------------------------------
------------------------------------------------------------
Snort % Pkts dropped and mbits/sec for the last 20 minutes
Dropped Packets = 0.000 Mbps = 4.672
Dropped Packets = 0.000 Mbps = 4.796
Dropped Packets = 0.000 Mbps = 4.369
Dropped Packets = 0.000 Mbps = 5.071
------------------------------------------------------------

Enjoy :)

MythTV Upgrade - Part 2

2008-02-18T12:34:00.005-05:00

Configuring lirc (Remote control daemon)

Getting the remote control to work has been on my to-do list for as long as I've been using MythTV. Early on I decided to go with a wireless mouse/keyboard combo instead. I have been using the Ione Scorpius P-20 for quite a while and it has served me well.

Every 6 months or so I would try to get the remote working, and every time I would fail... until this weekend. I couldn’t have done it without the following two sites:

1) http://www.mythtv.org/wiki/index.php/MCE_Remote
2) http://www.hauppauge.co.uk/board/showthread.php?t=8048

I have a Hauppauge PVR-150 Tuner card which came with the Remote and the IR receiver. I would say my biggest stumbling point along the way was that until this weekend I never knew exactly which remote I had. Apparently the PVR-150 has come with a whole bunch of different remotes over its lifetime.

As it turns out, I have a MCE USB2, Version 2, Hauppauge PVR-Kit remote. How I was supposed to know that without luckily finding that first link, I have no idea. A picture truly is worth a thousand words sometimes. The only identifying number on the back was "RC 6" ir.

So now on to the configuration:

MythDora4 comes with lircd version 0.8.2-CVS:

[root@mythtv mythtv]# /usr/sbin/lircd -v
lircd 0.8.2-CVS

After reading a bunch of caveats on the MythTV.org wiki link above, I decided to use CVS and go right to the latest version.

I following the wiki instructions exactly:

407 cd /usr/src
408 cvs -d:pserver:anonymous@lirc.cvs.sourceforge.net:/cvsroot/lirc login
409 cvs -z8 -d:pserver:anonymous@lirc.cvs.sourceforge.net:/cvsroot/lirc co lirc
410 cd lirc
411 ls
412 ./autogen.sh
413 ./setup.sh
Menu Option # (1) - Driver Configuration (enter)
Menu Option # (8) - USB Devices (enter)
Menu Option # (t) - Windows Media Center Remotes (new version, Philips et al.) (enter)
Menu Option # (3) - Save your configuration and run configure (enter)
418 make && make install
419 modprobe lirc_mceusb2

This installed lircd in /usr/local/sbin/lircd (This will be important later). First I used mode2 to see if the IR receiver was working:

[root@mythtv lirc]# mode2
(I then pressed the up arrow on the remote)
space 100000
pulse 2750
space 750
pulse 500
space 400
pulse 500
space 350

- That output means it was catching the signals

Unfortunately when I started lircd, ran irw, and then pressed the same buttons, nothing showed up:

[root@mythtv lirc]# /usr/local/sbin/lircd /etc/lircd.conf
[root@mythtv lirc]# irw

I tried a few pre-made lircd.conf files online, but the one from that second link is what finally worked. The lircd.conf file is what maps the IR code to a button on your remote. It looks something like this:

Power 0x00007bf3 # no e2,e3
MyTV 0x00007bb9 # starts at af
MyMusic 0x00007bb8 # starts at af
MyPictures 0x00007bb6 # starts at af
MyVideos 0x00007bb5 # starts at af
Record 0x00007be8 # no e2,e3
Stop 0x00007be6 # no e2,e3
Pause 0x00007be7 # no e2,e3

This time, when I run irw i see the following output:

[root@mythtv lirc]# lircd /etc/lircd.conf
[root@mythtv lirc]# irw
000000037ff07be1 00 Up mceusb
000000037ff07be0 00 Down mceusb
000000037ff07bdf 00 Left mceusb
000000037ff07bde 00 Right mceusb
000000037ff07bfa 00 Five mceusb
000000037ff07bf9 01 Six mceusb

That is farther then I have ever been before. Now the last part is the .lircrc file. This file maps the named button to the program and action (or keystroke) and is located in the users home directory. (ex: /home/mythtv/.lircrc)

To review:

Lircd.conf -> Maps IR code to a button name on remote
.lircrc -> Maps the button on the remote to a corresponding keystroke (application dependent).

An excerpt of the .lircrc file looks something like this:

# Down = Scroll/Channel Down.
begin
prog = mythtv
button = Down
config = Down
repeat = 2
end

This tells us that when (according to lircd.conf) the down button is pressed, if we are in mythtv, this should be equivalent to the down arrow.

I then started lircd with my new lircd.conf and .lircrc files in place:
[root@mythtv lirc]# lircd /etc/lircd.conf

And with that my remote control worked with MythTV for the first time ever.

The last thing I did was to change the /etc/rc.d/init.d/lircd file so that the service script starts my newly compiled lircd .0.8.3-CVS rather than the stock lircd.

Just to reiterate which version is which:

[root@mythtv mythtv]# /usr/sbin/lircd -v
lircd 0.8.2-CVS
[root@mythtv lirc]#/usr/local/sbin/lircd -v
lircd 0.8.3-CVS

I added the text in red to the following file: /etc/rc.d/init.d/lircd

[ -x /usr/local/sbin/lircd ] exit 1
[ -x /usr/local/sbin/lircmd ] exit 1

start(){
if [ -f /etc/lircd.conf ]; then
echo -n $"Starting infrared remote control daemon: "
daemon /usr/local/sbin/lircd $LIRCD_OPTIONS
RETVAL=$?
echo
fi

Troubleshooting:

The licrd binary that came with MythDora4 wrote debug information to /var/log/messages. On the lircd I complied myself, it wrote message to /var/log/lircd. Tailing (with a -f) whichever log file lirc is writing to can be a really good way to troubleshoot.

MythTV Upgrade - Part 1

2008-02-03T10:15:00.005-05:00

Introduction

I've been using MythTV for about 3 years now, both on Fedora Core and also on Ubuntu on my laptop. My first MythTV system was built with A LOT of help from Jarod Wilson's infamous How-To. A few months ago I built two MythTV systems for my family and decided to use MythDora4. It was so quick and easy that I decided to use MythDora for my own rebuild as well.

I should start off by saying that I don’t use MythTV for the PVR functionality. I use it solely as a digital jukebox. I watch TV shows and movies using MythVideo, MythMusic is always a big hit at parties, and occasionally I use MythImage for slideshows.

In the past I had all my media files on my WindowsXP box and used cifs to mount the windows shares on my MythTV box. I played music/videos directly through the share and performance was great even with a 10/100 Mbps NIC.

For this iteration, I decided to also upgrade to a 1.5TB RAID5 array so that I could start burning all of my DVD's to .ISO files. This way I can browse through my entire DVD collection digitally.

Lastly, I recently bought a Harmon/Kardon receiver and Polk Audio Center channel and Bookshelf speakers and slowly realized that I wasn’t going to be able to appreciate the new hardware unless I got digital audio working. (And yes… it is amazing)

Below, are my notes, impressions, etc for the entire setup. I am going to put as much detail as possible so that if I ever have to do this again I have it all in one place.

Hardware

Dell 8300 Tower, Dell BIOS A07
CPU: P4 2.8Ghz Hyperthreaded
Memory: 1.25Gb PC3200
Audio: M-Audio 5.1 Audio Card
Video: Nvidia GeForce 5200 AGP (VGA/DVI/SVIDIO)
Capture Card: Hauppauge PRV-150 Tuner Card
Network: 10/100/1000 Intel NIC
Storage:
(1) 40 GB HDD (IDE) (for the OS)
(2) 500GB IDE HDD's (for the RAID)
(2) 500GB SATA HDD's (for the RAID)

Configuring the OS

The install of the OS is really simple and extremely intuitive. The problems I ran into were caused by my additional disks, but were a result of a bug/lack of code/oversight/etc in the Dell BIOS. The motherboard has two IDE controllers, and two onboard SATA controllers. My buddies at work gave me the thumbs up on a RAID array that included two IDE and two SATA drives so that’s what I went ahead and did. I already had one 500 IDE, so I went and bought one more 500GB IDE and two 500GB SATA drives. This was good advice, and to fast forward, I did get it working, but Dell was not going to make it easy for me.

After tons of research trying to prove that I wasn’t crazy, I confirmed that on this Dell 8300 BIOS, if you use either or both of the SATA controllers, the BIOS will only let you boot off a SATA drive. This means that if I installed the MBR (Master Boot Record) on one of my IDE drives, no matter what I did in the BIOS, I could not boot off of it. Most BIOS’s would obviously let you choose whichever disk you would like, but of course that would make too much sense.

The most obvious option was to just install the MBR onto one of the SATA drives. However the whole point of using a separate system disk is that I want to make the RAID array and the OS completely independent of each other. If the disk that had the MBR on it died, I would be out of luck.

A more elegant solution to this problem which was proposed by my coworker, is to instal the MBR on both of the SATA disks, so that if one died, the other one would just pick up, but like I mentioned earlier, I wanted to keep the RAID seperate.

So what I finally decided to do was to install the MBR on a 128 MB USB disk. The end result is kind of a convoluted setup, but I think it’s a pretty cool solution. I like the cool factor that my machine won’t boot without that thumb drive in place, and of course this keeps my RAID array completely separate from the OS. Speaking of cool... I shortly found out that 5 drives in that stock Dell case was the farthest thing from it, but that will be a completely seperate article :)

So back to the installation. While installing the OS I choose to make my own partitioning scheme, which looked like this:
/boot partition on sdc (the USB disk) using all 128MB
/boot1 partition on hda (100MB)
/ partition went to hda as well (my 40GB system disk).
For all three I used the ext3 filesystem.
swap went on hda as well (2048 GB)

I then installed the OS, rebooted, and FINALLY saw that wonderful line “Grub loading… Please wait” or whatever it is.

So for this first boot both phase I and II of the boot loader took place on the USB drive. To play around I wanted to also see if I could get the actual kernel to boot from the system disk rather than the USB disk. To do that I had to do the following:

I edited the grub/grub.conf that was on the USB disk to look like this:

default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title MythDora-hda (2.6.20-1.2944.fc6)
root (hd5,0)
kernel /vmlinuz-2.6.20-1.2944.fc6 ro root=LABEL=/ rhgb quiet
initrd /initrd-2.6.20-1.2944.fc6.img
title MythDora-usb (2.6.20-1.2944.fc6)
root (hd0,0)
kernel /vmlinuz-2.6.20-1.2944.fc6 ro root=LABEL=/ rhgb quiet
initrd /initrd-2.6.20-1.2944.fc6.img

I then copied the following files from /boot/ to /boot1:
config-2.6.20-1.2944.fc6
initrd-2.6.20-1.2944.fc6.img
symvers-2.6.20-1.2944.fc6.gz
vmlinuz-2.6.20-1.2944.fc6
System.map-2.6.20-1.2944.fc6

I then edited /etc/fstab to look like this:

LABEL=/ / ext3 defaults 1 1
/dev/hda1 /boot ext3 defaults 1 2
Devpts /dev/pts devpts gid=5,mode=620 0 0
Tmpfs /dev/shm tmpfs defaults 0 0
Proc /proc proc defaults 0 0
Sysfs /sys sysfs defaults 0 0
LABEL=SWAP-hda2 swap swap defaults 0 0
/dev/cdrom /media/cdrom auto noauto,ro,user 0 0

Then I rebooted again. This time, grub still ran from the USB disk, but it looked at hd5,0 for the kernel, which on my machine is /dev/hda1. It found it and then mounted /dev/hda1 to /boot.

I am not sure which is better: Leaving the entire boot partition on the USB drive, for using this phased approach; but this is the way I left it. Let me know if you think it makes a difference.

The only current problem with the actual OS (MythDora4) is that atrpms recently deprecated the FC6 package repository so updating the system is kind of at a standstill until the next version of MythDora comes out. Luckily, http://g-ding.tv/?q=content/what-were-working, shows that the next release based on Fedora Core 8 will be coming soon. They are just waiting on the release of version 0.21 of MythTV.

So I'll leave off with that. Parts two and three will include the following topics:

Configuring the storage (RAID5 + XFS)
Configuring Digital Audio (S/PDIF over coax using alsa)

Still on the task list it to get the following working:

Configuring LIRC (Remote control)
Configuring VGA to Component Video

Hello

2008-01-31T15:06:00.001-05:00

So this is going to be my technical brain dump space on the web. As someone who has spent thousands of hours of my life asking Google how to do stuff, I finally feel like I am rounding the curve to the point where I can give something back.

I would guess that 98% or more of this blog will be related to open source Linux projects. Some of the ones I use the most are Snort, MythTV and Drupal. Most of my entries will be pretty basic and for my own records, but I hope that every once and a while someone else will find one of my entries useful and it will help them the way that so many amazing pages have helped me in the past.