Category Archives: python

Komodo IDE auto-generating setters/getters for PHP

I’ve got about thirty auto-generated PHP Doctrine models that are missing their required Java style setters/getters. At class number three of typing in these accessory methods I snapped and said “There has to be a better way” and magically the universe smacked me upside the head and reminded me that I’m using Komodo IDE which just so happens to have a disgustingly powerful Python powered macro system.

Four minutes later, out came the copy & paste hacked together monstrosity below. It’s not perfect but it doesn’t need to be, just has to work well enough to save my sanity and my client’s time.

To use, follow Komodo’s help documentation for creating a new python Macro then copy and paste this code into the macro window OR a slightly easier way, make the macro then click the “edit macro” context menu property to open the macro source file as a new view in Komodo.

The macro uses some very simple rules. It’s only looking for private properties in a format of “^\s*private \$[a-zA-Z0-9_]$”, it collects all of these variable names and then appends them through the setter and getter templates to a string buffer. Finally the buffer is
inserted into the current document at the position of the cursor. The output is coherent but not whitespace friendly which isn’t too big of a deal to re-format. Note, the macro has no concept of PHP syntax, so if there is more then one class in a file, the results will not be desirable.


from xpcom import components
import re

viewSvc = components.classes["@activestate.com/koViewService;1"]\
    .getService(components.interfaces.koIViewService)
view = viewSvc.currentView.queryInterface(components.interfaces.koIScintillaView)

sm = view.scimoz
sm.currentPos   # current position in the editor
sm.text         # editor text
sm.selText      # the selected text
#sm.text = "Hello World!"

output = u"\n"

setterTemplate = """
    function set%s($value){
        $this->%s = $value;
    }
"""

getterTemplate = """
    /**
    *@return string
    */
    function get%s(){
        return $this->%s;
    }
"""

propertyTemplate = """
%s
    
%s
"""

prefixSize = len(u"private $")

def formalName(rawName):
    return u"%s" % "".join([part.title() for part in rawName.split("_")])
        



#todo find a better way to split lines, what if its Mac or Windows format?
for line in sm.text.split("\n"):
    if line.strip().startswith("private $"):
        #trim of the private $ and trailing semi-colon
        realName = line.strip()[prefixSize:-1]        
        output += propertyTemplate % ( setterTemplate %(formalName(realName), realName), getterTemplate % (formalName(realName), realName))        
        
    
    
sm.insertText(sm.currentPos, output)
        

Semi-critical OpenSSL memory issue shared with Node.js and Twisted Python

I found this ( http://journal.paul.querna.org/articles/2011/04/05/openssl-memory-use/ ) just a tad disturbing. The TLDR is that OpenSSL automatically sets compression on for SSL connections. That’s great if you’re trying to cut down on bandwidth… but not so great when thinking about both Memory and CPU utilization, definitely more so memory in this case.

Fortunately the post author and company created some partial duct tape solutions for anyone suffering under out of memory issues in their infrastructure.

Python pydoc module

Python documentation here

Example

$python -m pydoc pydoc

Talk about eating your own dogfood! This is exactly like using the help() function in the python command line interpreter… except accessible from your shell prompt…. but wait!

It’ gets better! Not only does it make julian fries ( may not for any implementation ) but it’s got a few versatile little secrets

$ python -m pydoc
pydoc - the Python documentation tool

pydoc.py  ...
    Show text documentation on something.   may be the name of a
    Python keyword, topic, function, module, or package, or a dotted
    reference to a class or function within a module or module in a
    package.  If  contains a '/', it is used as the path to a
    Python source file to document. If name is 'keywords', 'topics',
    or 'modules', a listing of these things is displayed.

pydoc.py -k 
    Search for a keyword in the synopsis lines of all available modules.

pydoc.py -p 
    Start an HTTP server on the given port on the local machine.

pydoc.py -g
    Pop up a graphical interface for finding and serving documentation.

pydoc.py -w  ...
    Write out the HTML documentation for a module to a file in the current
    directory.  If  contains a '/', it is treated as a filename; if
    it names a directory, documentation is written for all the contents.

Now the graphical interface isn’t anything to write home about, but the -p option provides a no thrills web interface to
almost everything accessible to your python interpreter. This can make it slightly easier to troll through foreign modules
looking for undocumented sub modules and classes… or having an accessible reference doc for properly managed modules

Python urllib

Python documentation here

Unfortunately there is little or no documentation on the command line properties of urllib but it does recognize everything that urllib can handle. So
python -m urllib http://website.com will grab the specified url and print to std out

Note FTP works as well but you need to follow the pattern ftp://user:password@website.com if authentication is required

Python module dis

Python documentation here

Example

python -m dis myFile.py provides an interesting look into a python file’s guts
I could easily imagine this being part of some sort of static time inspect system where dis sits at the front and a parse
walks down the output lines, turning the data into a dependency and symbol graph. Unfortunately it doesn’t seem to provide anything more and is really just a test function most likely intended for unit-testing the python stdlib.

Python SimpleHTTPServer

When working on pure javascript applications ( canvas widgets & such ), I’ve found using the SimpleHTTPServer disgustingly useful as it serves the current working directly without much thrills.

Python documentation here

Usage

$ python -m SimpleHTTPServer 8081 0.0.0.0
Serving HTTP on 0.0.0.0 port 8081 ...

Note that it’s not necessary to set the 2nd argument to 0.0.0.0 if you want the service to listen on all routes. It normally will by default listen on everything… just habit for me to always append that.

Another useful part of this server is that it servers an apache directory style listing of all file’s present unless there is a valid index file like index.htm present.

Python json.tool module

Python documentation here

Example usage:

$ echo '{"json":"obj"}' | python -mjson.tool
{
    "json": "obj"
}
$ echo '{ 1.2:3.4}' | python -mjson.tool
Expecting property name: line 1 column 2 (char 2)
   

All argument patterns are:

  • piped json string | python -m json.tool which syntax checks then outputs the results to stdout
  • python -m json.tool input_file.json which reads the path relative file and outputs the results
  • python -m json.tool input_file.json output_file.json only difference here is that the output is directed to the specified file

My thoughts, this could be part of some sort of data validation check, looking for corrupted json static files.

Example

$ echo '{"a":123, "foo":"bar" }' | python -m json.tool && echo "IS valid" || echo "Is not valid"
{
    "a": 123,
    "foo": "bar"
}
IS valid
$ echo '{"a"1:123, "foo":"bar" }' | python -m json.tool && echo "IS valid" || echo "Is not valid"
Expecting : delimiter: line 1 column 4 (char 4)
Is not valid

Python – batteries included

Inspired by this
AND here

I’m going to do my best to either write up some examples of how to use these or link to someone/somewhere else on the
internet where someone did a better job then my grammar handicapped self can

For those to lazy to click the above liniks, the list below is a semi-complete list of command line accessible modules
to perform utility work.

So python -m calendar prints out a pretty calendar of the year, much like the GNU linux cal command line function.

  • json.tool -> pretty prints JSON Examples
  • SimpleHTTPServer -> serve the current directory over HTTP on port 8080 Examples
  • quopri / uu / binhex / base64 -> encode / decode Quoted-Printable / UUEncoded content.
  • telnetlib -> ghetto telnet client
  • filecmp -> directory entry comparison tool
  • ftplib -> ghetto FTP client
  • smtpd -> SMTP proxy
  • timeit -> command line profiling interface. Very handy
  • calendar -> prints year calendar
  • urllib -> ghetto wget Example
  • zipfile -> ghetto info-zip
  • aifc -> dumps some info about the provided aiff file (if given two paths, also copies path1 to path2)
  • cgi -> dumps a bunch of information as HTML to stdout
  • CGIHTTPRequestHandler -> same as SimpleHTTPServer except via the CGIHTTPRequestHandler: it will executes scripts it recognizes as CGI, instead of just sending them over (has not survived the transition to Python 3)
  • compileall -> compiles a tree of Python files to bytecode, has a bunch of options. Does not compile to stripped files (pyo)
  • cProfiler -> runs the provided script file (argument) under cProfiler
  • dis -> disassembles a python script Example
  • doctest -> runs doctests on the provided files (which can be python scripts or doctest files)
  • encodings.rot_13 -> rot13 encodes stdin to stdout (has not survived the transition to Python 3)
  • fileinput -> some kind of ghetto pseudo-annotate. No idea what use that thing might be of
  • formatter -> reformats the provided file argument (or stdin) to stdout: 80c paragraphs &etc
  • gzip -> ghetto gzip (or gunzip with -d), can only compress and decompress, does not delete the archive file
  • htmllib -> strips HTML tags off of an HTML file
  • imaplib -> ghetto IMAP client
  • locale -> displays current locale information
  • mimify -> converts (mail) messages to and from MIME format
  • modulefinder -> lists the modules imported by the provided script argument, and their location
  • pdb scriptfile.py -> automatically enters PDB post-mortem mode if the script crashes
    platform, displays the platform string
  • poplib -> dumps a bunch of info on a POP mailbox
  • profile -> see cProfile
  • pstats -> opens a statistics browser (for profile files)
  • pydoc -> same as the pydoc command Example
  • sgmllib -> see htmllib (as far as I can tell)
  • shlex -> displays tokenization result of the provided file argument (one token per line prefixed with Token:)
  • SimpleXMLRPCServer -> XMLRPC server for power and addition
  • telnetlib -> telnet client
  • tokenize -> dumps tokenization result of a Python file
  • webbrowser -> opens provided URL in your default web browser (options: in a new window, in a new tab)
  • whichdb -> A helpful little tool to try and tell which DB-api driver to use on a specified DB file