Python Crib Sheet


 

See http://docs.python.org/2/tutorial/index.html

 

 

Features of Python 2.7 no longer supported in Python 3 are struck out.

 

Getting Started

 

To run scripts from the windows console:

 

Add python to the path in environment variables: Control Panel > System > Advanced > Environment Variables > System Variables > Path : Edit

 

Assuming that python.exe is in C:\Python33\ and your scripts are in C:\Python33\scripts\  … To the end of the path add ;C:\Python33; C:\Python33\scripts

 

Type the script name at a windows command prompt to execute it.

 

To run scripts from the python command line:

 

Add your script path to the environment variables: Control Panel > System > Advanced > Environment Variables > System Variables  : New

 

Add a new variable called “PYTHONPATH” and provide the script path as its value; e.g. “C:\labs”

 

At windows cmd prompt type “python”.

 

This gives you an interactive prompt, >>>.

 

You can now import the script: >>> from myScript import *

 

On first import, any code not in a function will be executed.

 

You can re-run the script using reload: >>>reload(myScript)

 

Now you can run functions defined in your script.

 

IDE

 

IDLE is the built-in IDE, and is much easier to use than a shell.

 

You can also develop in Eclipse, Visual Studio etc.

 

PyCharm for Windows / Mac / Linux - highly recommend. Allows separate development of projects each with their own Python and library versions.

 

Launch IDLE, the basic IDE for Python:

 

Ctrl-N              Create script in new windows

 

Ctrl-S              Save the script, not forgetting to add .py extension.

 

F5                    Run script.

 

Alt-P/Alt-N    Retrieve previous / next line at the >>> prompt.

 

Tab or Ctrl-]    Indent On selected lines. Ctrl-[ to back-indent

 

_                      Underscore represents the result of the previous command.

 

F1 or help() gets interactive help. Q to quit help. Ctrl-z to quit python.

 

Basics

 

#                                           comment.

 

;                                            statement separator for multiple statements on one line.

 

=                                           This is not assignment as we know it! It means make LHS a reference to the RHS.

 

+=,-=, *=, /=,%=         augmented assignments but not ++ or --. a+=b is not the same as a=a+b

 

a+=b modifies a (if it is mutable). a=a+b reassigns a to new value, a+b.

 

/                   Result is int if operands are int, float if one is a float.

 

//, **                                integer division, exponentiation

 

== , !=             comparison: equality and inequality

 

 

 

and,or,not          short-circuited logical operators

 

‘,”, ‘’’            Quotes may be nested. Triple-quote allows multi-line strings.

 

a is b, id(obj)      Test identity of references, return address.

 

0x, 0o, 07          0x1AbC for hex literals, 0o7 for Octal literals. 07 octal (2.7)

 

if myVar > 1:       condition  brackets are optional

 

      statement blocks             by indentation, not {..}

 

elif 0<a>=b:        can chain comparisons

 

   pass  empty statement

 

else:

 

   c = input(‘A?:’) local variable c remains in scope after the statement.

 

print(‘:’, c)        print to default, normally the console or IDLE window.

 

print ‘:’, c         (2.7)

 

 

 

returnIfTrue if condition else returnIfFalse       # Conditional expression, like ? in C.

 

 

 

while condition:      There is no do … while or until. No switch … case statement.

 

   continue         Jump to next loop iteration

 

   break            Exit entire while statement

 

else:               Execute when condition fails, except for a break.

 

   sys.exit(message) Terminate the process

 

 

 

for i in collection:    # Read-only iteration through entire collection. i is copy of element.

 

for i,m in enumerate(collection): # tuple of index & member of entire collection.

 

for i in range(start, end):     # serves up a list of integers which can be used for indexing.

 

 

 

def myFunction(arg1,arg2 = 3): # default args can be supplied from the right

 

   “expected arguments are…” # help for the function

 

   localA = 2               # locals are created by assignment. Scope is the function.

 

   print(someGlobal)            # unassigned variables must exist in a surrounding scope

 

   global b = 6             # assigning to a global requires global qualifier

 

   arg2 = 0                 # arguments are passed as copy references – safe to reassign

 

   arg1[1] = 0              # but arguments may be modified via the references

 

   return localVar          # return is optional

 

with                        # with is not for method selection but for context managers

 

 

 

Difference between += and = .. +

 

a=[5]                       # a is a mutable list

 

c=a                             # c is a copy reference to the same list

 

a+=[6]                      # a is modified

 

print('a,c ',a,c)

 

>>> a,c  [5, 6] [5, 6]      # c sees that a has been modified  

 

a=a+[7]                     # the ‘equivalent’ operation in long-hand

 

print('a,c ',a,c)

 

>>> a,c  [5, 6, 7] [5, 6]   # a gets reassigned. c remains unchanged.

 

Import

 

Modules are searched for in sys.path, which can be modified by sys.path.append(dir) or changing the environment variable PYTHONPATH

 

import module, MyModA                          use module objects by name – eg. MyModA.myFn()

 

from MyModA import myFn            use  myFn()

 

from module import *                            all names imported from module – eg. myFn()

 

import MyModA as myA          use  myA.myFn()

 

import AllMyModules.MyModA

 

import .sibling (from current package)  or import ..uncle from parent package.

 

Modules can be placed in directories, known in Python as a package.

 

Variables

 

Everything is an object, including integers, function definitions, class definitions and class instances. Any object, including a function call with particular arguments values, can be assigned to a reference name.

 

Variable names are untyped references to objects. Type is known by the object not by the variable. Any object can be passed to a function, as long as it supports the methods called by that function (known as duck-typing – if it waddles and quacks call it a duck).

 

type(myVar) returns the type of the referenced object.

 

Variables are created by assignment: myVar = something and are local to the enclosing scope unless qualified as global: global myVar = something

 

Assignment does not copy, it just binds the name to the object. Thus A=B makes A refer to the same object as B, it does not copy B into A. A=6 binds A to a memory location containing 6. A=7 rebinds A to some other location containing 7.

 

When a variable is being assigned to, its reference is being modified. Whenever a variable appears in an expression, you get a copy of the reference.

 

R = [a,b]  # copies of the references a,b are placed in the list R.

 

Copying a mutable collection should be done either:

 

myCopy = original[:]    # shallow copy with a slice if it does not contain nested collections

 

import copy;                     # deepcopy if it contains nested collections

 

myCopy = copy.deepcopy(original).

 

Numbers, strings and tuples are immutable. Thus characters in a string and elements of a tuple cannot be reassigned. Pass number arguments as single-element lists to make them mutable. Use slices and concatenation to create new strings:

 

myInt = [1]; myString = myString[0:2] + newChar + myString[2:]

 

Lists, Dictionaries & Sets are mutable.

 

Cast between types : str(myObj); int(myObj); float(myObj); list(myObj) etc.

 

Collections

 

Iterators are pointers to a collection member which can be incremented and decremented. They are returned by many collection methods including for loops. Lists can be created from iterators:  myList = list(myIterator) # The iterator is incremented until it completes the collection it points to.

 

Strings are immutable collections of one-byte ASCII characters for 2.7 or Unicode characters (default two bytes, internally compressed to one for ASCII). They can contain escape characters, e.g. \t, \n, \r, \’, \”,\0,\\. (escape is a back-slash – you escape backwards). A new-line can be escaped with \ at the end of the line in which case the following line reads as if it continued without a new-line. A string prefixed with r is a raw string which does not interpret \ as an escape character. E.g. r‘s\t’ contains ‘s\t’ not s plus tab.

 

  int(‘1456’); str(1456) ; ord(‘w’); chr(45)

 

Bytes are immutable arrays of byte integers.  Prefix string literal with b to create Bytes.

 

  myByte = b"Hello World"

 

  print (chr(myByte[1]))

 

   string.encode() and bytes.decode() to convert between strings and Bytes / Bytearrays.

 

Tuples are immutable collections of arbitrary types defined by () and/or one or more commas:

 

  myTuple = 1,; myTuple = (1,2); anonymous tuples: x,y = 1,2; x,y = y,x

 

Range([start,]stop) returns a list (2.7) iterator(P3) of integers which can be used to initialise anonymous tuples : intA, intB, intC = range(3) # 0,1,2

 

xrange([start,]stop) returns an iterator (2.7).

 

Lists [] are mutable collections of arbitrary types:

 

    myList = [a,1,”hello”]   or   list(a, 1,”hello”  or    list(myIterator)

 

Bytearrays are mutable arrays of bytes:

 

  myByteArr = bytearray(myString.encode())

 

  myByteArr[6] = ord('w')

 

  

 

Dictionaries {} are mutable paired unordered collections with unique keys

 

myDict = {mykey1:val1, ‘key2’:val2, ‘key3’:val3}

 

dict(key1=val1, key2=val2, key3=val3)

 

NOTE: when defined in {braces}, key must be a string var or literal in quotes. When using dict() the keys are effectively the names of arguments and do not take quotes and cannot be variables.

 

Add to a dictionary by assignment to a new key:    myDict[newKey]= newValue

 

Sets {} are mutable unique unordered collections. &,|,-,^ (exOr) operators.

 

    mySet = { val1, val2, val3 } or set(val1, val2, val3)

 

Concatenate and repeat Strings, Tuples and Lists may be joined or repeated with + and *:

 

     bothStr = myStr + yourStr; longList = shortList * 5

 

Sort: newSeq = sorted(oldSeq); returns a sorted list from any sequence.

 

myList.sort(key=sortFn) # Sorts lists in-situ. sortFn is applied to each element. Defaults to sorting elements by value.

 

Index collections from the beginning [0] or the end [-1]. myList[3]; myList[-3]

 

Dictionaries and Sets are indexed by the key:  mySet[someKey]

 

Membership: in, any(), all() short-circuited test for collection membership:

 

   if member in collection: # e.g. if ‘a’ in ‘happy’:

 

   if all(collection): # tests members for true

 

   if all(member == 0 for member in collection):

 

   if any(member != match for member in collection):

 

Zipping: Columns can be returned from parallel arbitrary-type collections using zip().

 

   for (x,y,z) in zip(setX,tupleY,listZ): returns an iterator of tuples containing the
             
print(x,y,z)                                        column values from the parallel collections.

 

Slice, Split, Join, Delete, Insert

 

[:] Slice returns a subset of any collection as the same type myCollection[start:stop:step]

 

myColl[1:2] (2nd element);   myColl[:2] (1st 2 elements);   myColl[:] (Entire copy)

 

myColl [-8::2]      returns alternate of last 8 elements.

 

myColl [-1:-8:-1]   returns elements last 7 elements reversed.

 

myList = string.split(separator, max_splits); returns list of sub-strings.

 

mySplit = ‘This-string’.split(‘-’); Args optional: separator defaults to space.

 

myString = separator.join(sequence);       separator may be ‘’

 

myString = ‘’.join(mySequence)

 

x,y*,z = myLongTuple        # x and z are assigned, the rest goes into y as a list.

 

Remove elements:

 

del myList[start:end] del myDict[‘key’]

 

removed = myCollection.pop(index[,default] Removes and returns a single element.

 

myCollection.remove(‘item’)           Removes the first matching element (it must exist).

 

Add Elements:

 

myList.insert(pos,item) ; myList[pos:pos] = [newA,newB]

 

myList += [newA,newB]

 

myList.extend([newA,newB])

 

myList.append(newA)

 

mySet.add(newA)

 

mySet |= sourceSet                              # set union with assignment

 

myDict.update(sourceDict)       # merge : add each member of sourceDict  to myDict

 

myDict.setdefault(key,default=none # return value if found, else default. Does not insert.

 

 

 

Functions

 

Python does not support function / method overloading. If you try it, you simply redefine the function as the last encountered definition. Use optional function arguments instead.

 

def myFn(argA,argB)     Using named args: myFn(argB=5,argA=3)

 

def myFn(*myArgs)          variadic argument, a tuple of arguments. Use: myFn(val1, val2)

 

def myFn(**myArg)          dictionary arg. Use: myFn(key1=val1,key2=val2,key3=val3)

 

def myFn(arg1,*,arg2): # ,*, forces all args to the right to be supplied by name (P3).

 

def myFn(arg1:‘list’):     # information about arguments. Is just a comment. Not enforced.

 

Local function variables must be assigned before use, otherwise they are treated as existing in a surrounding scope. To assign to a global variable, it must be brought into scope in the function body with global keyword, e.g. global myGlobalVariable = 1.

 

An anonymous string at start of function def creates a help entry. Triple-quote for multi-line.

 

Initialisation of mutable default arguments

 

def myFn(arg,listArg = [1]):

 

    listArg.append(arg)

 

    print(listArg)

 

 

 

myFn(2) # [1,2]

 

myFn(3) # [1,2,3] !!?? Why is listArg not defaulted to [1]?

 

The surprising behaviour above occurs because functions and classes are ‘constructed’ the first time they are encountered as the script executes. myFn is constructed with listArg pointing at a list [1].  The default argument reference is now fixed. If the defaulted object type is mutable and the function modifies that object, then the next call to the function will see a modified object at that pre-set default reference. We need a way to test if the defaulted argument has been supplied and if not manually apply the default. We can easily do this by setting the default to None instead of []. We can then test the argument against None and manually set the required default value:

 

def myFn(arg,listArg = None):

 

    if listArg == None: listArg=[1]

 

    listArg.append(arg)

 

    print(listArg)

 

 

 

myFn(2) # [1,2]

 

myFn(3) # [1,3]

 

Lambda

 

A lambda is a one-line function which can be substituted where a function is expected or assigned to a variable to give it a name. Any expression without assignments, branches or loops may be used. The last result of the expression is returned. lambda is simply a way of saying “What follows is a function definition”

 

returnVal = lambda arg1,arg2: someExpression

 

max = lambda a,b: a if a>b else b  # assigning a lambda to a name

 

result = max(1,2)                    # calling the named lambda

 

myList.sort(key=lambda str:int(str))  # in-situ lambda as sort function  

 

Functions as variables

 

A function is a piece of code which is referenced by its name. This is important for understanding function variables, closures and decorators.

 

So a class method can be aliased: myFn = myObj.someMethod

 

myFn is strictly a reference to a copy of the code defined in the class. We are not usually explicit in saying that every variable is a reference to such and such. The important point is that myFn is a copy of the code, not a reference to the original.

 

Use a lambda to assign a function with specific arguments to a variable:

 

cube = lambda x: power(x,3)

 

Nested Functions and Closures

 

Functions may be nested and can see the variables and arguments in the enclosing function. They cannot be called outside the enclosing function, but may be used as return values from the enclosing function. To enable a nested function to assign to a variable in the enclosing scope, we must qualify it as nonlocal (P3). 

 

When you return a function you get a piece of code back which you can assign to a variable.

 

def outer(x):

 

   outerMsg = “Outer”

 

  def inner():

 

       nonlocal outerMsg # Only in P3.

 

       outerMsg = “Inner” # modifies outerMsg rather than creating a new local

 

       print x

 

   inner()

 

   print(outerMsg)

 

  return inner

 

The above outer function returns its inner function code as its return value. Notice that the inner function prints the argument passed into outer. What happens if we call the inner function we get back from outer?

 

First we note that inner() cannot be called without first calling outer. The only way we can get access to inner is via the return value from outer. And outer must be called with an argument. The second thing to realise is that what is returned from outer is not a reference to the code inside outer but a copy of that code, with the argument x inserted. This behaviour is called a closure. It refers to the fact that a nested function returned from an outer function remembers the state of the arguments and local variables it could see when it was returned (it encloses the surrounding state). There is no mystery to this behaviour if you remember that what you get back is simply a copy of the code, as it looked when the function was returned.

 

A closure is a basically a function with state. The state is supplied by its enclosing function and is set when that function is called. It is the Python equivalent of a C++ functor or Lambda with capture variables.

 

def generate_power_func(n): # a meta-function; it generates a function.

 

    def nth_power(x):

 

        return x**n

 

    return nth_power

 

Note that the outer function returns a function which happens to be nested within the outer function. The returned function depends upon a variable, n, which is obtained from the outer function.

 

We use our generate_power_func by saying what power we want to use it with:

 

raiseTo4 = generate_power_func(4) # we now have a power-function

 

print(raiseTo4(2)) # >>> 16     # and we use it in anger.

 

This works because raiseTo4 is a copy of the nested function with the value n set to 4.

 

Decorators

 

The closure above takes a value argument, but we can write closures that take a function as the argument and return the inner function as the return value. We call this a decorator.

 

A decorator is simply a function that takes a function as an argument and returns a function. It effectively modifies the behaviour of the original function, and thus “decorates” it. It can be used in place of the original function.

 

def myDecorator (func):

 

    def myDecInner(*args, **kwargs):

 

        print "decoratedFunction"

 

        return func(*args, **kwargs)

 

    return myDecInner

 

We can for instance say:

 

foo = myDecorator(foo)

 

bar = myDecorator(bar)

 

Now, anyone using foo() gets the original function, modified by myDecorator. Likewise bar() has also been redefined by passing it through our decorator. Of course, we don’t have to redefine foo and bar, we could assign to other names, making both the decorated and original functions available. The point is that what comes back from calling our decorator is a copy of the inner function with the decorator modifications.

 

@decoration

 

The @ symbol is a piece of syntactic sugar that signals to the compiler to substitute a function with its decorated version, as we did explicitly above.

 

@myDecorator               

 

def aFunction(args):

 

    print "inside aFunction"

 

 

 

The above code says “Please substitute aFunction with  the result of passing it to myDecorator.”

 

When the compiler passes over this code, aFunction() is compiled and the resulting function object is passed to the myDecorator code, which produces another function object that is substituted for the original aFunction(). Anyone calling AFunction() will actually get the decorated version.

 

Decorators thus allow you to inject or modify code in functions or classes. @ decoration passes a function object through another function and assigns the result to the original function.

 

Namespaces and scope

 

A namespace defines the scope in which names are visible. Names propagate into scopes, unless hidden by the same local name, but do not escape out of scopes. Names are searched in:

 

  1. the innermost scope containing the local names
  2. the scopes of any enclosing functions, containing non-local, non-global names
  3. the current module’s global names
  4. the namespace containing built-in names

 

If a name is declared global or nonlocal then those outer variables may be modified. Otherwise, all variables found outside the innermost scope are read-only (an attempt to write to such a variable will simply create a new local variable in the innermost scope, leaving the identically named outer variable unchanged).

 

String Formatting

 

See http://docs.python.org/release/3.2/library/string.html#format-specification-mini-language

 

The format method is called on a formatting string, passing in the arguments to be formatted.

 

   ‘pre-text1{format1} pre-text2{format2}’.format(var1,var2)

 

The format part is in the form:

 

 {varPos : fillChar  align<>^  force+  #b/o/x  pad0  minWidth  ,.decPlaces  conversion}

 

e.g. print(‘Is {0:+#x}’.format(12))     # prints Is +0xc

 

print(‘Is {0:.^+15,.2f}’.format(12876)) # prints Is ..+12,876.00...

 

print(‘Is {:^015s}’.format(‘12876’))    # prints Is 000001287600000

 

print(‘Is {:^.15.3s}’.format(‘Hello’))  # prints Is ......Hel......

 

Each part is optional, default conversion being string - s.

 

For string variables,  fillChar of 0 and pad0 have the same effect.

 

For #b/o/x : binary/octal/hex  you cannot specify minWidth so fill/pad have no effect.

 

force+/-/space, # , (comma separator)           only apply to number variables.

 

. decplaces specifies digits after the . for f/F numbers or how many characters are used from the variable for string variables.

 

Conversions are:

 

          For string variables:           s        or nothing.

 

          For integer variables:         c        as character

 

                d       or nothing, as base 10 (decimal) integer

 

                                                    o        as octal

 

                                                    x/X    as lower/upper case hex

 

                                                    n        as number – decimal as set by locale.

 

          For decimal variables:        e/E    lower/upper case exponent

 

                                        f/F     fixed point

 

                                        g/G    or nothing – general format

 

                                        n        number – general as set by locale.

 

                                        %      percentage

 

Old-style formatting format_string % (variable,list)    Uses same formatting symbols

 

print(‘Is %+#x’%(12))      # prints Is +0xc

 

print(‘Is %+15d’%(12876))  # prints Is          +12876

 

print(‘Is %15s’%(‘12876’)) # prints Is           12876

 

Regular Expressions

 

Extract text that matches a pattern.

 

import re

 

Usually prefix pattern with r to get raw-text which ignores normal escape characters.

 

matchObject = re.match(r“pattern”,string [,flags]) # match start of string

 

matchObject = re.search(r“pattern”,string [,flags]) # match anywhere in string

 

changedString = re.sub(r“pattern”, r“replacement”, string [,count,flags])

 

changedString, count = re.subn(r“pattern”, r“replacement”, string [,count,flags])

 

stringList = re.split(r“pattern”, string [,max_splits,flags]) # str.split() is quicker

 

stringList = re.findall(r“pattern”, string [,flags])

 

stringIterator = re.finditer(r“pattern”, string [,flags])

 

print(matchObject.group())

 

print(matchObject.groups()) # if parentheses used to group patterns

 

 

 

.                       match any single character

 

[a-zA-Z]          match any char in the […] set. May include escape chars, e.g. \t

 

[^a-zA-Z]       match any char not in […]. ^- can be \ escaped, all else in [ ] are literals  

 

x?                    match 0 or 1 occurrences of x

 

x+                   match 1 or more occurrences of x

 

x*                    match 0 or more occurrences of x, greedy (stop at last match)

 

x*?                  match 0 or more occurrences of x, lazy (stop at first match)

 

x{m,n}            match between m and n x’s, greedy

 

x{m,n}?          match between m and n x’s, lazy

 

abc                  match abc

 

abc|xyz            match abc or xyz

 

(x)                   precedence or a capture group

 

\n                     back-reference to capture-group for use in replacement pattern in re.sub

 

 

 

Character Class Shortcuts – consume characters

 

For the purpose of Character Class Shortcuts and anchors, a word-character consists of letters, digits and underscore only. Note that ‘ is not a word character so a match for \w+ will return “don” when searching “don’t”. To get the whole word we would need to add ‘ to the search range [\w’]+

 

\w \W              match a word character / not a word character

 

\d \D               match a digit / not a digit

 

\s \S                 match a space / not a space

 

Anchors – do not consume characters

 

^ (if at start)    match beginning of text

 

$ (if at end)     match end of text

 

\b \B                word-boundary / not word-boundary

 

 

 

text = ‘the dog eats the delicious dog-food’

 

match from the first the and the last dog :

 

m = re.search(r‘the.*dog’,text)

 

print(m.group()) >>> ‘the dog eats the delicious dog’

 

match from the first the to the first dog :

 

m = re.search(r‘the.*?dog’,text) # ? makes * lazy

 

print(m.group()) >>> ‘the dog’

 

match all words starting with d:

 

mList = re.findall(r‘\bd\w*+’,text) # word boundary, d , any number of word chars

 

print(mList) >>> ['dog', 'delicious', 'dog']

 

Wild cards (Shell Metacharacters)

 

Although similar to re metacharacters, ‘wildcards’ apply to expanding file names in a unix shell, whereas re’s apply to matching patterns in text within a file. Confusingly, Microsoft uses wildcard syntax for text pattern matching.

 

[a-zA-Z]          match any char in the […] set. May include escape chars, e.g. \t

 

[!a-zA-Z]        match any char not in […]. !- can be \ escaped, all else in [ ] are literals   

 

?                      match any single character. NOTE re’s use ‘.’

 

*                      match 0 or more characters. NOTE In re, * applies to the previous character.

 

abc                  match abc

 

{a,b}               match a or b

 

a*                    match anything starting with a

 

*a                    match anything ending with a

 

Lazy lists, Comprehensions and Generators

 

We often only want part of a list. Rather than load the entire list into memory we can obtain a subset. This could be a list, or more efficiently an iterator. A lazy list uses iterators to return only the values we ask for.

 

filter(myFn,fromList) applies myFn to each item returning a list (2.7) iterator (P3) of items returning true. map(myFn,fromList) is a related function, but it returns every member applying myFn to each one.

 

A generator is an iterator on a list. It can be created with a special for-loop syntax:

 

generator = (member for member in collection if test(m)) (the if part is optional).

 

A tuple, list, dict or set can be created from an iterator, such as you get from a generator:

 

myTuple = tuple(generator)

 

A collection populated by a generator is called a comprehension. The generator may be defined in mutable collections (tuple cannot be created this way):

 

newList = [myFn(thisElement) for thisElement in fromList if myTest(thisElement)]

 

To get the item pointed to by the generator, use the built-in next(generator[,default])

 

Each call to next moves the iterator on.

 

A generator can be returned from a function, moving through a collection one element at a time whenever it is called. This can be done by providing a generator in the return statement, or by using a yield statement within a loop. Each call executes the next run round the loop, returning the next item. A generator can be used anywhere an iterator is expected.

 

def fibinachi(stop):

 

    a, b = 0, 1

 

    for i in range(stop):

 

        yield a             # yield makes this function a generator

 

        a, b = b, a + b

 

 

 

myList = list(fibinachi(10)) # create a whole list from an iterator

 

for x in fibinachi(10):     # or iterate with a for loop

 

    print(x)

 

myGen = fibinachi(10)       # or assign the generator to a variable

 

print(next(myGen))          # then next returns the result of the next iteration

 

print(next(myGen))

 

Iterators, as returned from a generator, have a __next__ method (next() 2.7) which move the iterator on to the next value. for calls __next__ for us.

 

Files

 

thisfile = open(“myDir/myfile.txt”[,’r/w/a’]) note: forward slash, unlike Windows!

 

          ‘r’   read-only (default)

 

          ‘w’   write-only – create or truncate existing.

 

          ‘a’   write-only – create or append existing.

 

          f e.g. ‘rb’ treats the file as binary rather than text. Use .encode() / .decode() on strings.

 

          f +     e.g. ‘rb+’ makes any option read & write.

 

myBuffer = thisfile.read(n)      # Read n characters, or the whole file.

 

myLines = thisfile.read().splitlines() # Return list of lines, discarding \n.

 

myBuffer = thisfile.readline()   # Reads a line, including the \n. Returns “” at end-of-file.

 

myLines = thisfile.readlines()   # Return list of lines, including the \n.

 

for line in thisfile:                                # returns an iterator, rather than reading the whole file.

 

     print(line,end = ‘’)      #  end= ‘’ stops print adding a \n since the line includes one.

 

outfile.writelines(myList)

 

outfile.flush()  # flush the buffer

 

myPos = thisfile.tell()     # return current position

 

thisfile.seek(myPos)       # jump to myPos. NOTE: seek confuses the iterator in a for loop!

 

thisfile.close()

 

Pickle

 

Pickling is the process of converting objects into byte streams for persistent storage (no encryption).

 

import pickle

 

pickle.dump(myObject,myBinaryFile.p)

 

myObject = pickle.load(myBinaryFile.p)

 

 

 

A shelve is a database for storing pickled objects by key which supports all dictionary methods. Each key exposes an object which is automatically unpickled and picked as you read / write the values.

 

import shelve

 

myDB = shelve.open(myDBfile.db) # creates it if file doesn’t exist

 

myDB[someKey] = newValue                   # adds new object if not found

 

oldVal = myDB[someKey]

 

myDB[‘myDict’] = myHugeDict               # add an entire dictionary as a single object

 

myDB.update(myHugeDict)        # add each key as a separate object (dictionary merge)

 

myDB.sync()                   # commits changes

 

myDB.close()                  # automatically performs a sync()

 

 

 

Unit Tests

 

Testing is built into python via doctest and docstrings

 

""" This is a sample module # multi-line docstring

 

    >>> today = Date(13,12,1949)     # >>> interactive session – no output

 

    >>> print today         # interactive session  followed by expected output

 

    13/12/1949

 

"""

 

if __name__ == "__main__":  # true if run as a program, not imported as a module

 

   import doctest

 

   doctest.testmod(verbose=True)     # run the tests

 

In IDLE the test will only print if it fails unless verbose set. To run the test from the cmd prompt: myModule.py -v

 

Class

 

A new style class (introduced in Python 2.2) inherits from object or type. Some new python features only work with new style classes.

 

Class myClass(myBaseClass): # base class is empty if not inheriting from a base class.

 

   myClassVar = 0           # class-wide data member, initialized first time only.

 

   def __new__(class,args): # not usually required, except for immutable classes whose

 

      pass                  # data must be set at construction.

 

   def __init__(self,baseArg):  # constructor. Destructor __del__ rarely used.

 

      myBaseClass.__init__(self,baseArg) # call base class initializer

 

      self.data = 0          # creating an object data member by assignment.

 

   def __add(self, x):   # double leading underscore:  private to the class (not enforced).

 

      self.data += x

 

   def addOne(self, x):     # defining a public method

 

      self.__add(1)         # calling private method via self.

 

   def __str__(self):       # operator overload. E.g. __add__, __eq__.

 

      return str(self.data)

 

   def usesBaseFn(self,arg):

 

      result = super().__baseFn(arg) # alternative syntax to call baseclass fn

 

 

 

__new__ is the first step of instance creation and is responsible for returning a new initialised instance of your class; it calls , __init__ for you. In contrast, __init__ doesn't return anything; it's only responsible for initializing the instance after it's been created. You shouldn't need to override __new__ unless you're subclassing an immutable type like str, int, unicode or tuple. Immutable classes must be initialised during construction via __new__.

 

Member functions take a minimum of one argument, the object reference, usually called self. When called on an object, the argument is automatically passed in, so myObj.fn() is equivalent to MyClass.fn(myObj)

 

When delegating to another member function within a member function, you must call that function on self. The interpreter then passes self in as the first argument for you:

 

def myClassFn(self):

 

   return self.otherFn()

 

 

 

_myObj : private to module. __myObj : private to class – but privacy is not enforced.

 

Class Attributes, Properties & Methods

 

An attribute is any member of a class – either data or a function, public or private, accessed with the dot notation: myClass.classMethod; myObject.objectData.

 

A method is a function defined within a class.

 

Data declared outside of a method is class data (i.e. shared by all objects) unless it happens to be assigned to a method, which makes it a member attribute!

 

Object data is defined by methods, usually __init__.  It is visible to the entire class.

 

Class myClass:

 

  def __init__(self,arg):

 

    self.__myData = arg         # create and initialise private object data

 

    self.pubAttribute = 2       # create a public attribute

 

  def getObjData(self):         # all methods take self as an argument

 

    return self.__myData        # access object data

 

myObj = myClass(5)              # create an object of myClass

 

myObjData = myObj.getObjData() # retrieve object data

 

print (myObj.pubAttribute)      # access object attribute

 

Object data can also be created on the fly for any instance simply by assigning to it! I.e. an object without a counter can be given one simply by myObject.counter = 1. But beware! Data attribute names override methods of the same name.

 

Properties are class attributes that look like data, but in fact are objects that wrap hidden object data in method calls. This type of object is called a descriptor and provides assignment and outward-conversion allowing us to use the assignment operator on that property which gets translated into method calls to the getter and setter methods:

 

a = myObject.myAttr                # calls a getter function on some hidden data

 

myObject.myAttr = b                 # calls a setter function on some hidden data

 

Properties can be created in the class using myAttr = property(myGetFn,mySetFn)

 

The Property constructor takes the getter and setter functions as arguments and returns a Property object, which we call a descriptor. It also has .setter, .getter, and .deleter functions allowing us to inject the functions that will be used for these operations on the attribute.

 

Properties are commonly created using the property class as a decorator:

 

class Date:

 

   ...     

 

   @property

 

   def mday(self):          # equivalent to property(mday(self))

 

      return self.__day    

 

                                                               # mday is now redefined as a property object.      

 

   @mday.setter             # equivalent to property.setter(mday(self,day))

 

   def mday(self, day):

 

      self.__day = day

 

 

 

Although this looks like function overloading it is actually function re-definition. Without the use of the property decorator, you could not make use of both mday functions. The second would simply redefine mday to be the second function.

 

Apart from assignment to properties as above, object assignment in Python is always assignment of an object to a reference, not assignment from one object to another. In other words, the left-hand side of an assignment is not the object it references, it is merely the reference itself. For this reason, although other operators can be overloaded for classes, assignment cannot.

 

Class Methods (C++ static method) :

 

   @classmethod

 

   def get_myData(class_name_place_holder):

 

      return myClass.__myData

 

 

 

   client = myClass.get_myData()

 

Class methods are passed the class as an argument and so can see class data.

 

Static Methods (helper functions) :

 

   @staticmethod

 

   def noData():

 

      return “I need an argument to do something with”)

 

 

 

   client = myClass.noData()

 

Static methods do not know if they are being called on an object, and can be called on the class or an object. They are helper methods associated with the class.

 

Delegation by composition with dependency injection

 

class proxy:

 

   def __init__(self, obj):

 

      self.__myObject = obj # inject the dependency on obj

 

   

 

   def __getattr__(self, objAttr):

 

      return getattr(self.__myObject, objAttr)

 

 

 

client = myProxy.doSomething() # calls doSomething on the injected object. The __getattr__ function is called whenever an attribute request is not provided by the class. It should call the built-in gatattr() function on the injected object to get the appropriate result.

 

 

 

Inheritance example

 

class person:

 

    __nextId = 0

 

    def __init__(self,name):

 

        self.__name = name

 

        self.__id = person.get_nextID()

 

    def getName(self):

 

        return self.__name

 

    def getID(self):

 

        return self.__id

 

    @classmethod

 

    def get_nextID(myClass):

 

        myClass.__nextId +=1

 

        return myClass.__nextId

 

 

 

class employee(person):

 

    def __init__(self,name,salary):

 

        self.__salary = salary

 

        person.__init__(self,name)

 

    def getSalary(self):

 

        return self.__salary

 

 

 

me = employee("Fred",100000)

 

print(me.getID(),me.getName(),me.getSalary())

 

 

 

you = employee("Bob",200000)

 

print(you.getID(),you.getName(),you.getSalary())

 

 


Exceptions

 

try:

 

       code

 

except exception_list as var:    # optional var holds tuple of error arguments

 

       handler code

 

except exception_list as var:    # stack unwound until matching except is found

 

       handler code

 

else:

 

       normal code

 

finally:                    # executed before stack-unwind if no matching except here.

 

       always executed code                       # do not assume that the try code succeeded

 

raise errorType(errorArgs)      # Raise an error. errorArgs is typically a message string.

 

class MyError(Exception):   # Create a new exception type.

 

   pass

 

Context Managers

 

Some classes have inherently paired operations which may raise an exception, such as file open/close and lock acquire / release. Such classes cab be written to perform the required try / finally code for you through executing special __enter__ and __exit__ methods. The use of such classes is simplified using with.

 

foo = file("/tmp/foo", "w")

 

try:

 

   print >> foo, "Hello!"

 

finally:

 

   foo.close()

 

is simplified to

 

with file("/tmp/foo", "w") as foo:

 

    print >> foo, "Hello!"