Saturday, April 28, 2012

Installing Django-nonrel Easily Automated

Notice: Development of this installer has been discontinued due to my personal discontinued use of django-nonrel. If anyone else would like to maintain this installer, contact me (sepero 111 @ gmx . com) and I will link to your work.

If you've ever tried setting up django-nonrel for Google App Engine, then you realize that it can be a pain. Multiple archives to download, long directories to rename, trying to figure what goes where, and the extreme tediousness of setting up multiple symbolic links...

This program will automate the process of installing Django-nonrel for GAE.

Download Link:
https://github.com/Sepero/django-nonrel-installer/archive/master.zip

You will need to run it on the command line with the argument of your desired django-nonrel branch. The available arguments are:
  • --master    to install django-nonrel 1.3 master (This option will be removed in future releases.)
  • --dev13      to install django-nonrel 1.3 development
  • --dev14      to install django-nonrel 1.4 development
  • --dev15      to install django-nonrel 1.5 development
  • --clean      to erase the current install

The installer should be run in the directory where it exists. Here is an example of how to install 1.5 development in a terminal 
$ python dj_nonrel_install.py --dev15

Installing 1.3 development in a terminal
$ python dj_nonrel_install.py --dev13

The folder "django-nonrel" will be created, and all files will be downloaded/installed within the folder for use with Google App Engine. After it's finished, just move/rename the directory to where ever you want the permanent location to be.

Speed up creation of your development environment and get on to making awesome django apps! Any improvement suggestions or submissions are very welcome. This is tested on Linux, and will likely work on similar systems. It most definitely will not (yet?) run on MSWindows. Should now run on all Operating Systems!


Download Link:
https://github.com/Sepero/django-nonrel-installer/archive/master.zip
https://github.com/Sepero/django-nonrel-installer


Reference Links:
Django Documentation - http://docs.djangoproject.com/en
Django-Nonrel Documentation - http://djangoappengine.readthedocs.org/en/latest/index.html
Django-Nonrel Mailing List - http://groups.google.com/group/django-non-relational




Update 2012 Apr 28:
Currently working to make the installer dual compatible with installing MongoDB also. (Use pip to install MongoDB.)

Update 2012 May 01:
Django-testapp doesn't work with MongoDB for some reason. A bug report has been filed with it's author. I'm sorry, but at the time of this writing, MongoDB support cannot yet be implemented.

Update 2012 May 15:
The installer has now been modified to work with all operating systems. For systems that don't support symbolic links, the libraries will be moved directly inside of the Django-Testapp folder.

Update 2012 May 28:
Support has been added for "master" and "develop" branches. Master is installed by default. To install development branches of django-nonrel, use the command line argument --dev.

Update 2012 Jun 4:
Added Reference Links.

Update 2012 Sep 20:
Fixed problem with downloading development branches.

Update 2012 Dec 25:
Branch 1.4 is now available to install. Changed to install all contents into a folder.

Update 2013 Mar 14:
Fix for djangoappengine library. Switched source hosting to github.

Update 2013 May 19:
Django-nonrel 1.5 now available to install. Fixed issue with urls and unittests. Updated program description.

Update 2013 May 20:
I've just been informed that "master" branches will retired and all branches will be considered development. The default install branch will be moving to 1.5 development.

Last Update 2013 Sep 24:
There will be no more updates of this installer as I no longer work with django-nonrel. You are free to take the code and continue work/updates on this project. Email me, or Leave a comment in the section below with a link to where others can find your work.


Comment or email your feedback at sepero 111 @ gmx . com

Friday, April 27, 2012

Django-nonrel pissing me off

Is it really so difficult to makes things relatively straight forward for the end user? The instructions for installing django-nonrel pretty much translate to "Just unzip the shit in a folder, dude! It's awesome *cue Bill and Ted air guitar music*"

Ok, so I'm exaggerating somewhat, but the truth is it's not awesome. Instead it's a convoluted pain in the ass. Multiple files to download and untar. Ridiculous directory names that all have to be renamed. Half the libraries are an extra directory deep, while half aren't. If you're a developer worth half your salt, you'll likely setup the symbolic links instead of just dumping all the directories together. Setting up the symbolic links is made even more annoying and tedious because of the multiple links necessary, and the "hide n seek" nature of the libraries. As a large part of forgiveness, I understand that the django-nonrel team aren't to blame for at least half these issues.


The solution? A program to automate the whole process, so you can get on with making awesome django apps! I'll be uploading my python program soon for the specific purpose of automating the install of django-nonrel. Hopefully it will help others out in getting jump started with an otherwise nice system.


How it works:
1. Create an empty working directory.
2. Put the python program in there.
3. Run it from inside that directory! *cue Bill and Ted guitar music*


Just that easy. It will: download the zip files->extract them->delete the zips->properly name the new folders->create all symbolic links->give you a hint to get going


Update:
Get it all here- Django-nonrel Easy Automated Installer
http://seperohacker.blogspot.com/2012/04/installing-django-nonrel-easily.html



Comment or email your feedback at sepero 111 @ gmx . com

Binary Grep Program: SearchBin

SearchBin is a fast commandline program for searching within binary files. It's a bit like grep for binaries.

It has three capabilities for searching.
-Search for bytes using hexidecimal
-Search for a plain text string
-Search for a smaller binary file


EXAMPLES
Search for the hex bytes "FF14DE" in the file gamefile.db:
$ ./searchbin.py -p "FF14DE" gamefile.db
Match at offset:            907          38B in  gamefile.db
Match at offset:           1881          759 in  gamefile.db
Match at offset:           7284         1C74 in  gamefile.db
Match at offset:           7420         1CFC in  gamefile.db
Match at offset:           8096         1FA0 in  gamefile.db
The printed offsets are listed in decimal and hex formats.


You can also search for unknown patterns with "??". Just insert them where ever you have an unknown byte:
$ ./searchbin.py -p "FF??DE" gamefile.db

You can search through multiple files at once, and search piped input:
$ ./searchbin.py -p "FF??EE" gamefile.db supersecret.idx
$ cat gamefile.db | ./searchbin -p "0xFF??EE"

You can also search using regular text strings and other binary files.
$ ./searchbin.py -t "hello" gamefile.db
./searchbin.py -f binaryfile gamefile.db 



Options of SearchBin:

$ ./searchbin.py --help

Optional Arguments:
  -h, --help            show help message and exit
  -f FILE, --file FILE  file to read search pattern from
  -t PATTERN, --text PATTERN
                        a (non-unicode case-sensitive) text string to search
                        for
  -p PATTERN, --pattern PATTERN
                        a hexidecimal pattern in format '0xFF'
  -b NUM, --buffer-size NUM
                        read buffer size (in bytes). 8MB default
  -s NUM, --start NUM   starting position in file to begin searching
  -e NUM, --end NUM     end search at this position, measuring from beginning
                        of file
  -m NUM, --max-count NUM
                        maximum number of matches to find
  -l FILE, --log FILE   write matched offsets to FILE, instead of standard
                        output
  -v, --verbose         verbose, output the number of bytes searched after
                        each buffer read
  -V, --version         print version information




Extra Notes:
An argument -t or -p or -f is required. The -p argument accepts a
hexidecimal pattern string and allows for missing characters,
such as 'FF??FF'. When using -f argument, the pattern file will
be read as a binary file (not hex strings). If no search files are
specified, %prog will read from standard input. The minimum memory
required is about 3 times the size of the pattern byte length.
Increasing buffer-size will increase program search speed for
large search files. All size arguments (-b -s -e) are read in decimal
format, for example: '-s 1024' will start searching after 1kilobyte.
Pattern files do not allow for wildcard matching.
Reported matches are displayed as 0-based offset.


Further Examples:
Search for the text string "Tom" in myfile.exe. Text is case sensitive.
./searchbin.py -t "Tom" myfile.exe

Search for the text string "T?m" in myfile.exe, where ? is a wildcard. This will match "Tom" "Tim" "Twm" and all other variations, including non-printing bytes.
./searchbin.py -t "T?m" myfile.exe

Search for the hexidecimal pattern "AABBCCDDEE" in myfile.exe.
./searchbin.py -p "AABBCCDDEE myfile.exe

Searches for the hexidecimal pattern "AA??CC??EE" in myfile.exe, where ?? can be any byte value.
./searchbin.py -p "AA??CC??EE" myfile.exe

Takes the binary file pattern.bin, and searches for an exact match within myfile.exe.
./searchbin.py -f pattern.bin myfile.exe





Features:
+No compiling necessary
+Requires Python 2.7 or Python 3
+Less code
+Search in files of unlimited size
keywords: hex hexidecimal binary like grep search seek find fast



DOWNLOAD it from here:
https://github.com/Sepero/SearchBin/archive/master.zip

Source:
https://github.com/Sepero/SearchBin

Report Problems, Suggestions, or Thanks to sepero 111 @ gmx . com
  or https://github.com/Sepero/SearchBin/issues




2012 Jun 19 Update:
Major over haul to search function. Dramatically increased search speed for wildcard patterns. Also included search functionality for regular text strings.

2012 Jun 28 Update:
Made updates to Readme file and added more comments to code for readability.
Also, I moved all files to Mercurial and bitbucket.com.

2012 Jul 07 Update:
Unifying all documentation. Publishing to freecode.com

2013 Feb 02 Update:
Switched code indentation to tabs as it is a universal standard. Moved code back to github due to its popularirty.

2013 Oct 11 Update:
Added Python 3 support. Added in more unittests. Added in more information code comments. Updated documentation.