Archive for the ‘unix’ Category

Unix shell Hackery

Saturday, February 4th, 2012

I was helping a friend try to set up a git repository of a rather large existing software project.  He loves guis, and the github gui tool was choking on the initial commit/upload.  Here is the series of shell commands I sent him via email to execute.  It’s pretty cool what you can do with unix and how it lets you communicate.

$EXT is the root directory where you have stored the existing code base

# copy everything over to a completely new directory
1. cp -r $EXT ~/th2

# cd into that directory
2. cd ~/th2

#how much space is that directory taking
3. du -h | tail

#remove the git part of that directory
4. rm -rf ~/th2/.git

#how much space is it taking up now
5. du -h | tail

# how many files are in this whole directory tree
6. find . | wc -l

# what are the most common file extensions,  change -n30 until you think it's giving a reasonable report
7.  find . -type f | grep -v git |   sed 's/\(.*\)\([\/.][^\/]*$\)/\2/'  | grep -v "^\/" | sort | uniq -c  | sort -nr | head -n 30

#create a file size by type script
echo "#/bin/bash" >> size_by_extension.bash
echo "size=`find . -type f | grep "$1$" | xargs -L1 ls -lrt  | awk '{sum+=$5} END{print sum}'`" >> size_by_extension.bash
echo 'echo "$size $1"' >> size_by_extension.bash

./chmod +x size_by_extension.bash

# what file types use the most space
9. find . -type f | grep -v git |   sed 's/\(.*\)\([\/.][^\/]*$\)/\2/'  | grep -v "^\/" | sort | uniq | xargs -L1 ./size_by_extension.bash  2> /dev/null  | sort -nr 

There may be more elegant ways to get some of this done, but all in all it is pretty cool. imagine trying to communicate a similar workflow with gui oriented commands.

My server was hacked

Wednesday, December 3rd, 2008

Last wednesday, I checked my blog and it had been hacked.  wp-header had been edited and borked.   Nothing in the database had been touched.  It was a scary experience, but I got the server back up quickly.  I backed up the database and restored it.  Apparently there were some encoding issues, thus the weird characters everywhere.  I updated to the latest version of wordpress and everything seems to be fine.

I think it was just a driveby attack, nothing aimed at me personally.  I have been working on blogging more and getting links to my blog from other sites.  Oh well.  Time to restore the db properly I guess.

stumpwm — wow

Thursday, September 18th, 2008

I had a long train ride today, and little battery life left.  So I read the manual for stumpwm and played with it.  Wow.  The real key for me was figuring out how to restore windows to full size after splitting them “C-t Q”.  So here are the commands that I use to navigate stumpwm
“C-t s” splits a window (actually a frame in stumpwm parlance)  vertically — making it half as tall

“C-t S” splits a window horizontally — making it half as wide (horizontal vs vertical splits may be obvious to some people but they always seem to trip me up when I read them).

“C-t f” puts a number in the upper left hand corner of each frame, pressing the number for the frame you want to go to will do that.
“C-t Q” makes a window full-screen
“C-t w” lists the windows — applications running

“C-t #” where # is the window you want to use, brings that window to the top of it’s frame and puts focus there

“C-t C-#” grabs a window from whatever frame it is in, and pulls it into your current frame

getting emacs, a terminal (for emacs to run in), and firefox were all major amounts of damage, that I have sustained for the last month or so.  Firefox was by far the hardest, when I get FF3 setup in x on OS X I will write about the process.

Here were links that helped me  — i’m still digesting this but it seems to be the msot thorough explanation of xmodmap and emacs key oddities that I have seen.

by using xev, I have figured out that I have 9 seperately addressable modifier keys available to me , counter-clockwise (capslock, l-shift,l-ctrl,windows,l-alt,r-alt, windows-context,r-ctrl,r-shift).  freaking sweet

Security Anomally in CoreGraphics Event Taps

Thursday, September 4th, 2008

When I was playing around with changing modifier keys for OS X I stumbled onto some inconsistencies with permission requirements and abilities for two very similar event types.

The code I used here is modified from . You can view the code on github here
There are two programs included in the tar ball.  getKeyPressCode and insertKeyPressOnModifier. They have a very similar structure.  I will start by describing getKeyPressCode

GetKeyPressCode has two functions, main and a call back function.  A call is made to CGEventTapCreate , which passes the myCGEventcallBack function in (I’m hazy on my c code,  it must somehow pass in a pointer), and event mask flags for the type of event, in this case kCGEventKeyDown or kCGEventKeyUp.
The callback in getKeyPressCode displays the keycode for any character you press, in any application on the system. If you press the = key, it replaces that with ‘f’ .
To run getKeyPressCode, you need to be superuser (sudo) or have Assistive devices enabled.  Otherwise the program will fail saying, event tap failed to create.  Also notice that when you go to a  systemwide password box (such as you would be prompted for in keychain) , no keyUp or keyDown events are fired, even though the program was run with superuser privileges.

The second program is insertKeyPressOnModifier.  This program has the same form, the difference is that it catches modifier keys — CGEventFlagsChanged.  This program displays the EventFlags for modifier keys (“CAPSLOCK” , “SHIFT”, “CTRL”, “OPTION”, “OPTION/ALT”, “APPLE”) when a modifier key is pressed.  When any modifier key is pressed, the program inserts an “=” or “+” depending on whether or not shift was pressed.  Again this behaviour takes place in any application on the system.

Now the interesting thing is, you don’t need superuser rights or assistive devices enabled to run insertKeyPressOnModifier.  Even more interesting is, inserKeyPressOnModifier still fires events when you are in a system wide password box, it will also insert ‘=’ characters.  This seems like a potential security hole.

I filed a bug report for this a year ago, apple hasn’t responded or fixed the hole.  This I have observed this behavior on a Mac Book Pro running Tiger and Leopard.  I’m not well versed in C, CoreGraphics, OS X internals, or general security measures like this.  There could be a very good explanation for the behavior, to me though, it seems like an inconsistency that could be a hole.  I haven’t seen the password box behavior mentioned anywhere else.

You can look at the header file for CGEvent on your mac here


modding bs.el for emacs

Wednesday, August 6th, 2008

I use bs.el to switch between buffers in emacs.  I have bs-cycle-next mapped to C-Tab and bs-cycle-previous mapped to C-Shift-Tab .  The problem is when I quickly page through my buffers, the buffer list that shows up in the mini window rotates and the current buffer doesn’t show up in the list,

1-2-3-4-5 c-tab

2-3-4-5-6 c-tab

the position of buffers doesn’t remain the same, which makes it hard to quickly go to the desired buffer.  I wanted behaviour like this

(6) 1 2 3 4 5

6   (1)  2  3 4 5

6   1  (2)  3 4 5

With the included file, bs.el now behaves in the way I prefer.   This is the first serious lisp programming I have ever done, also the first mod I have made to emacs.

TODO: figure out font-face-lock to get the current buffer highlighted instead of surrounded with pipes ||

The nvidia article that actually helped me

Saturday, March 15th, 2008

Some hope for subversion on the mac

Tuesday, January 30th, 2007

As I alluded to in a previous post, it is possible to integrate subversion into finder with applescript.  In fact someone has already done this SVN scripts for Finder  .  I haven’t yet played with these, but they are open source, so I hope to contribute.

Publicly available subversion host

Tuesday, January 23rd, 2007

Subversion is crucial for my development efforts, without I spend a lot of time getting frustrated with things that should be easy.  Subversion is very useful even when I’m the only coder on a project, it is a necessity for collaborative development.   I wanted to get a subversion server that I could access anywhere in the world and that clients could access, they can’t get to my linux box sitting behind my cable modem.

I looked at getting a virtual private server, you get a good amount of space with them, but you have to setup subversion yourself.  Dreamhost now offers subversion hosting too, but I have heard mixed reviews about them.  Version control isn’t something I wanted to skimp on.  I ended up going with CVSdude hosted subversion.

I’m really glad I did, I pay less than a VPS for something that is up right now.  For $30/month I get unlimitted repositories (each client will get their own repository), and 5 Gigs of total space.  I also get web-svn, bugzilla, VC, and Trac project management.  Each repository gets its own trac setup.  All of this is managed with a decent web interface (it’s table based :( ), and it just works.   I probably saved at least 2 or 3 days of sysadmin work by going with CVS dude.  Comparative advantage is a great thing to remember.
The CVS Dude service has been good.  They have a good pipe, I get 40K up, and 600-900k down, I’m pretty sure my cable modem is the limit, those are about the most I get for uploads and downloads for other activities.  I misnamed my repository initially, thinking I was picking my admin name, instead that was the base name for all my repositories.  A short email to their support fixed that in a couple hours.  I am concerned about looking professional with a repository URL that looks like  luckily I can get a domain name for my repository and they say it will work.