_______     _______ _______________
`\     `\ /'     /' `\           /'
  \  |`\ ' /'|  / ()  \  |`,    / ()
   | |  \ /  | | ____  | |  \   |____
   | |   |   | | \  /  | |   |  |\  /
  /   \     /   \ ||  /   \  |  \ ||
./_____\. ./_____\__\/_____\ |___\__\             ____
                            /|                   `-,  |
                          __||__     ___     ___    | |     ___
                         `-,  ,-'  ,',-,`, ,',-,`,  | |   ,',--`.
                            ||     | | | | | | | |  | |   \`===,
                            ||__,. | |_| | | |_| |  | `-, .____)|
                            `---'   `---'   `---'   '----` `---'


Intro
-----

Currently small collection of random tools I've written that may be useful to
others.

They're mostly for my personal use but there's nothing stopping them from
working elsewhere and they're reasonably stable, but may not catch all possible
error states or sanitize input, so be careful when processing untrusted data.
They'll all likely be command line programs so make sure you know how to use a
command line, but some are simple as running in the same directory or you could
even just drag a file in to it.

Python scripts may require at least python 3.x unless otherwise noted.


Tools Included
--------------
extoggs.py
  Extracts OGG Vorbis files from Blockland's cache.db.  Just run it to get
  information on using it.

dc2iso.py
  Reads a GDI file and outputs a ISO.  Specifically designed to work with
  Dreamcast images, though hasn't been widely tested so might not work on all
  images or ones with oddball protections.  Files output with this utility won't
  work immediately, however, they may need their LBA fixed.  Requires python 3.4.

fixlba.py
  Takes an ISO file and an LBA offset and adjusts all pointers within the file.
  LBA can be adjusted forward or backward with a negative argument.  Paired with
  the previous program, you can make mountable Dreamcast images that you can
  browse with any file manager.

unscale
  Graphical utility to help with unscaling a scaled image.  Not very useful,
  just a little experiment I worked on to see about recovering an approximate
  image of retro gaming footage from videos or screenshots that have been scaled
  up.  Seems to mostly work well with hugely upscaled images
  (low res 256x224 -> 720p/1080p), not likely to work well with lesser scale
  factors.  Only takes BMP, requires SDL 1.2.  TIP: You might lose a pixel on
  each side.

  usage: unscale filename.bmp [xsize [ysize [destw [desth]]]]
    xsize, ysize - initial window size, window is resizable at runtime
    destw, desth - initial destination surface size, how large you think the
                   original image may be

  Controls:
    Q          - quit and print final params
    P          - print current parameters to terminal
    M          - sampling mode (topleft, center)
    W,S        - adjust source vertical start
    A,D        - adjust source horizontal start
    T,G        - adjust desired height
    F,H        - adjust desired width
    I,K        - adjust source vertical range
    J,L        - adjust source horizontal range
    Up,Down    - adjust display vertical scale
    Left,Right - adjust display horizontal scale

  Sampling Mode
    This program will only do nearest neighbor scaling so unless that's what you
    want, it's not intended to be your final output.  The intention is to take
    the parameters you got from this program and feed it in to another program
    with more capable scalers.  However, different programs will sample from
    different places given the same parameters.  When scaling down an image, a
    certain amount of pixels are to be stepped over for each result pixel.
    Topleft mode starts at the actual source start position every time.  Center
    mode will start at that position plus half the step required per result
    pixel, taking the center sample for that result pixel.  If your output looks
    fuzzy when you pass the parameters to another application, try changing
    sampling modes and refocusing the image with the WSAD source start
    adjustments until it looks clear, you shouldn't have to adjust range however
    you may find that you need to slice off a pixel from each side, then 
    readjust range.

fix_bl_save.py
  Might help to fix weirdly messed up blockland save files where certain bricks
  will be offset by 1 in various directions.  Tested and should work with most
  default bricks.  Doesn't consider any custom bricks.  Uses included 
  BLSFile.py also written by me.

ytanno2ass.py
  Converts a YouTube annotations XML file in to an Aegisub (.ASS) subtitle file
  for viewing along with a video in a media player.  Has a few issues.  The
  font width calculation requires a Tk context which opens a window for a brief
  period.  Translucent or unfilled rectangles don't seem to work.  Not sure if
  it's a limitation of libass or my syntax is off.  Various effects aren't
  implemented.  Cards aren't implemented.  Style isn't 100% correct.  Speech
  bubbles don't have the little triangle pointer thing.  Font metrics don't
  match those used in the YouTube flash player, nor match those used by the
  actual subtitle renderer so some wrapping issues will come up.  Otherwise,
  should more or less work well enough for most YouTube annotations to the
  degree that they should at least be readable.  Written for Python 2.7, should
  work in 3.x but untested.
  
blockland-colorset-tool.py (PaletteCore.py, BLPalette.py, GPLPalette.py)
  Converts between blockland colorSet.txt and GIMP Palette format for easier
  editing in something like mtPaint.  Unfortunately, GIMP palette format doesn't
  store alpha, and is unable to store the palette categories, so an extra file
  is output along with the GPL file.  If you're going to rearrange colors or
  change divisions, you should choose not to restore extra data and adjust alpha
  directly in the colorSet.txt file, or modify the extra data file manually.
  The format is as follows (all values are ascii alphanumerics):
  first value is the number of colors
  following values go for the number of colors, each one is the alpha value of
  each color in the order they appear in the color set.
  next value is the number of divisions
  following values are the locations of the divisions, then after that is the
  names of the divisions
  
  Run the script with the -h or --help parameter to get more information on
  usage.

ps3swiz.c
  Unswizzles textures used in some PS3 game I don't know the name of.  Uses a
  "nested tile" based swizzle pattern.  Doesn't properly support non-square
  textures.  If someone wants that working, I can try to look in to it.

  build with:
    $ gcc `pkg-config --libs MagickWand` `pkg-config --cflags MagickWand` \
    -ggdb -o ps3swiz ps3swiz.c

  usage:
    $ ./ps3swiz <u|s> inputfile outputfile
    
    'u' will unswizzle, 's' will swizzle.

rsfgen.py, extractcci.sh, makecia.sh
  Tools utilizing ctrtool and makerom from Project CTR to assist in converting
  dumped 3DS cartridges in to CIA packages.  The makecia.sh and extractcci.sh
  scripts are just examples of usage.  They need to be improved to be made more
  universal.  rsfgen.py probably needs python 3.2 or greater.
  
  usage:
  $ python rsfgen.py path/to/image.cci path/to/exheader.bin
  
  RSF file will be output to stdout.  You can use whatever your shell provides
  to pipe that to a file (usually the '>' operator) or just copy and paste to a
  text file.

videonow_dec.c, videonow_encode.sh
  Decode videos ripped from VideoNow discs.  It requires that you concatenate
  tracks together for features that span across tracks.  Usually works well
  enough if you concatenate all tracks from a disc in to 1 file.  Playback can
  be pretty rough due to there being far less redundancy in audio tracks, so
  make sure you get a perfect rip for best results.  The program will try to
  resync if there are problems but it can only do so much.
  
  The program takes a raw CD audio track in little endian byte order as output
  by `cdparanoid -Bp` on a little endian platform.
  
  The videonow_encode.sh script is just a small example for how to use ffmpeg to
  encode the videos in to something reasonably watchable.

256to
  Converts Marathon .256 chunks extracted from the Mac Shapes resource fork
  data.  Outputs indexed TGA files.  For some reason they don't seem to work in
  The GIMP but imagemagick can open them, so you can always just convert them to
  something else.  Make it with 'make', use it by passing a file to it.  It'll
  output a bunch of data for you as well.  process.sh will process all files
  named *.raw in the current directory (you'll need to fix the path to 256to.)
  and put the output of each in to its own directory with the output saved to a
  report.txt.  cleanup.sh will just delete all these, but you should be careful
  to just run it within the same directory it's in with nothing else, because it
  just deletes all *.tga, *.txt files and all directories within the directory
  it is run in.

3dmazeto
  Another texture ripper.  This one much simpler.  It will convert some of the
  non-BMP resources from the 3D Maze screensaver (Maybe others, I don't know
  what program this format is associateed with...) in to TGA.  Like the last
  one, the TGA files don't open in the GIMP so just convert them to PNG or
  something that supports alphas with imagemagick convert.  The signature for
  these files is "A3 A2 A1 A0" starting at byte 0.

barex
  Extract Prey Evil .bar files.

ps.py
  Updated version that displays a bit nicer, and works with larger numbers of
  CPUs/cores/threads, up to 36.
  Displays a graph of CPU usage per core, and displays the most heavily loaded
  core to the end, filling the unused time in the graph with /es.  This will
  try to display a useful range, as indicated by the percentage on the right.

grpex
  Extracts all files from a Duke Nukem 3D .GRP file.  Had trouble finding one
  that I could use that was simple and compiled readily so I wrote one.  It's
  very basic and has few features other than extracting all files.

artex.py
  Extracts all Duke Nukem 3D ART found in a directory.  Requires a PALETTE.DAT
  and will import a NAMES.H to give useful filenames to tiles.  Writes files out
  as paletted BMPs.

export_palettes.py
  Extracts palettes from a Duke Nukem 3D PALETTE.DAT and LOOKUP.DAT and writes
  them all out to .txt format.  They import to mtPaint, maybe others, to apply
  the game's alternate palettes to its art.

blocktext.py
  Outputs text in a dumb blocky font to the terminal.
  USAGE: ./blocktext.py "<text>"
  All text must be in 1 argument, so if it's multiple words or would otherwise
  be split by the terminal, it should be quoted.  Also only supports the
  printable 7 bit ASCII character codes.

  If no arguments are given, read from stdin.

uncdk
  Deobfuscates CDK videos from some Blockbuster CD.  I imagine other things
  used it.  Seems to be some macromedia director thing so maybe some common
  thing for throwing together promo materials.  Just compile it, then run it
  on the file you want to deobfuscate.  It does the operation in-place so make
  sure to back it up in case it doesn't work.

iomemdec.py
  USAGE: ./iomemdec.py [-h] <filename>

  Reads and displays a potentially useful view of /proc/iomem by displaying
  the start and size of each entry.

  The filename can be /proc/iomem if run as root or otherwise with privilege
  to read the values from /proc/iomem, otherwise you'll just get all 0s.

  -h can be passed for more human-readable sizes (KiB, MiB, GiB, TiB).

vid2aud.py
  USAGE: ./vid2aud.py <file> [file] [file] ...

  ULTRA simple converter from video to audio, mostly good for youtube and
  probably not _really_ complete for this purpose.  It'll put vorbis and opus
  in to .ogg and aac in to .m4a.  Even if it's just an audio file, it'll still
  remux it in a hopefully most-compatible container for portable device.

  Requires ffmpeg, ffprobe.

make-convergence-hook.py
  USAGE: ./make-convergence-hook.py

  Outputs a shader hook for mpv which will produce a shader for adjusting
  RGB convergence/alignment.  Made to adjust out oddities in an old CRT TV
  without having to get in to the TV itself.  This is very raw so you pretty
  much need to adjust the parameters in the code.  The default size is 4x4 but
  with some adjustment, any size can probably be used.

godot_object_scanner.py
  USAGE: ./godot_object_scanner.py check <filename>
         ./godot_object_scanner.py list <filenmame>
         ./godot_object_scanner.py dedup <filename> [filename]

  check
   Tries to check linkage of the file and for references to missing items.
   All the other operations will start with this operation.

  list
   List all "paths" to objects (resources, nodes).

  dedup
   Determine duplicate resources and fold them in to one.  This will also
   deduplicate any intentionally "unique" resources!  Maybe at some point
   there may be a way to add exceptions.  If no second filename is given, it
   will only list the duplicates, otherwise it will also output to a new file,
   however, it will try not to write to the same file as a backup should be
   kept in case it ruins the file or does something unsavory.

gmod_map_list.py
  USAGE: ./gmod_map_list.py [--list | --sort[=]<criteria[,criteria,...]> | --dump | --steampath[=]<path to steam> | <workshop ID>]...
         ./gmod_map_list.py --collisions-scan [--steampath[=]<path to steam> | --threads[=]<number of threads>]...

  Sort criterias:

  id : ID
  size : Size
  updated : Last Updated Time
  published : Published Time
  name : Name
  author : Author
  files : File Count
  maps : Map Count

  List all gma files and information provided by them from the steam workshop
  directory.  A list of workshop IDs can be given to get information on only
  specific items.

  --list
   Enable reduced mode that just lists ID, name and map files.

  --dump
   Extract all files to disk.  Probably want to specify specific IDs otherwise
   it'll extract all of your addons.

  --steampath
   Provide the path to the directory steam is installed in, default is
   .local/share/steam in whatever python determines is the home directory.

  --collisions-scan
   List all files from addons that are shared with other addons or mounted
   game data.  This works on a lot of assumptions of how mounted content works
   so it might not be super accurate, but it might be a useful guideline to
   maybe narrow down some things.

  --threads
   Only for collisions scanning mode.  Number of threads to use for hashing,
   default is 1.  4 improves things for me, 8 just a bit more, but it's
   probably diminishing returns from there unless maybe you have a very fast
   SSD, but I suspect most of the slowness is python itself.

dn1gfx.py
  USAGE: ./dn1gfx.py <filename>

  Install:
   virtualenv <path to venv>
   source <path to venv>/bin/activate
   pip install pillow

  Convert a Duke Nukum/Nukem 1 DN1 tile or background or cutscene graphics
  file to an indexed PNG.  Requires pillow.

  Will output a JSON with the header data too.  Giving it back the JSON with
  the PNG present will return back a DN1.  The PNG should still be indexed
  with the same palette in the same order (most graphics packages which are
  good for handling indexed graphics should respect this).