CWPK #21: Some Accumulated Tips

• When using a cell in Markdown mode for narratives, it is sometimes useful to be able to add HTML code directly. A nice Jupyter Notebook WYSIWYG assistant is:
  • https://github.com/genepattern/jupyter-wysiwyg; install via:

    
    
    conda install -c genepattern jupyter-wysiwyg 

    However, after a period of time I reversed that position, since I found using the assistant caused all of the cell code to be converted to HTML vs Markdown. It is actually easier to use Markdown for simple HTML formatting
• I tend to keep only one or two Notebook pages active at a time (closing out by first File → Save and Checkpoint, and then File → Close and Halt), because not properly closing a Notebook page means it is shown as open next you open the Notebook
• When working with notebook files, running certain cells that cause long lists to be generated or large data arrays to be analyzed can cause the notebook file when saved to grow into a very large size. To keep notebook file sizes manageable, invoke Cell → Current Output → Clear on the offending cells
• When starting a new installment, I tend to first set up the environment by loading all of the proper knowledge bases in the environment, then I am able to start working on new routines.

or type:

Directories and files

Any legal directory or file name is accepted by Python. For Windows, there is often automatic conversion of URI slashes. But non-Linux systems should investigate the specific compatibilities their operating systems have for Python. The differences and nuances are small, but can be a source of frustration if overlooked.

Here are some tips about directories and files:

• Don’t put a """comment""" in the middle of a dictionary listing
• A """comment""" header is best practice for a function likely to be used multiple times
• When reading code, the real action tends to occur at the end of a listing, meaning that understanding the code is often easier working bottom up, as references higher up in the code are more often preliminary or condition setting
• Similarly, many routines tend to build from the inside out. At the core are the key processing and conversion steps. Prior to that is set-up, after that is staging output
• Follow best practices for directory set ups in Python packages (see CWPK #37).

Modules and libraries

A module name must be a valid Python name, limited to letters, digits and ‘_’s.

Modules are the largest construct in Python programs. Python programs consist of multiple module files, either included in the base package, or imported into the current program. Each module has its own container of variables. Variable names may be duplicated across modules, but are distinguished and prevented from name clashes by invoking them with the object (see earlier discussion about the ‘dot notation’). You can also assign imported variables to local ones to keep the dot notation to a minimum and to promote easier to read code.

Namespaces

Python is an object-oriented language, wherein each object in the system has a name identifier. To prevent name conflicts, Python has a namespace construct wherein any grouping of existing object names may be linked to a namespace. The only constraint, within Python’s naming conventions, is that two objects may not share the same name within a given namespace. They may share names between namespaces, but not within.

The namespace construct is both assigned automatically based on certain Python activities, and may also be directly set by assignment. Import events or import artifacts, like knowledge graphs or portions thereto, are natural targets for this convenient namespace convention.

When items are declared and how they are declared informs the basis of a namespace for a given item. If it is only a variable declared in a local context, it has that limited scope. But the variable may be declared in different ways or with different specified scope, in a progression that goes from local to enclosed to global and then built-in. This progression is known as the LEGB scope (see next).

All of this is logical and seemingly straightforward, but what is required by Python in a given context is dependent on just that: context. Sometimes it is difficult to know within a Python program or routine exactly where one is with regard to the LEGB scope. In some cases, prefixes are necessary to cross scope boundaries; in other cases, they are not. About the best rule of thumb I have been able to derive for my own experience is to be aware the the ‘dot notation’ hierarchies in my program objects, and if I have difficulties getting a specific value, to add or reduce more scope definitions in the ‘dot notation’.

To gain a feel for namespace scope, I encourage you to test and run these examples: https://www.programiz.com/python-programming/namespace.

Namespaces may relate to Python concepts like classes, functions, inner functions, variables, exceptions, comprehensions, built-in functions, standard data structures, knowledge bases, and knowledge graphs.

To understand the Python objects within your knowledge graph namespace, you can Run the following cell, which will bring up a listing of objects in each of the imported files via its associated namespace:

You’ll see that each of the major namespaces (sometimes ontologies) list out their internal objects as imported. You may pick any of these objects, and then inspect its attributes:

You can always return to this page to get a global sense of what is in the knowledge graph. Similarly, you may import the cowpoke package (to be defined soon!) or any other Python package and inspect its code contents in the same manner. So, depending on how you lay out your namespaces, you may readily segregate code from knowledge graph from knowledge base, or whatever distinctions make sense for your circumstance.

LEGB Rule

The scope of a name or variable depends on the place in your code where you create that variable. The Python scope concept is generally presented using a rule known as the LEGB rule. The letters in the acronym LEGB stand for Local, Enclosing, Global, and Built-in scopes. A variable is evaluated in sequence in order across LEGB, and its scope depends on the context in which it was initially declared. A variable does not apply beyond the scope in which it was defined. One typical mistake, for example, is to declare a local variable and then assume it applies outside of its local scope. Another typical mistake is to declare a local variable that has the same name as one in a broader context, and then to wonder why it does not operate as declared when in a broader scope.

Probably the safest approach to the LEGB scope is to be aware of variables used in the core Python functions (‘built-ins’) or those in imported modules (the ‘global’ ones) and to avoid them in new declarations. Then, be cognizant that what you declare in local routines only apply there, unless you take explicit steps (through declaration mechanisms) or the use of namespace and dot notation to make your intentions clear.

Setting Proper Data Type

Within owlready2, classes and properties are defined and treated as Python classes. Thus, when you retrieve an item or want to manipulate an item, the item needs to be specified as a proper Python class to the system. However, in moving from format to format or doing various conformance checks, the representation may come into the system as a string or list object. Knowing what representation the inputs are compared with the desired outputs is critical for certain activities in cowpoke. So, let’s look at the canoncial means of shifting data types when dealing with listings of KBpedia classes.

From Python Class to String

Much of the staging of extractions is manipulating labels as strings after retrieving the objects as classes from the system. There is a simple iterator that allows us to obtain sets of classes, loop over them, and convert each item to a string in the process:

 new_str_items = []
 for item in loop:
     a_item = item.curResource             # Retrieves curResource property for item
     a_item = str(a_item)                  # Converts item to string                
     new_str_items.append(a_item)          # Adds to new string list

If you have nested items within loops, you can pick them up using the enumerate in the loop specification.

From Sting to Python Class

The reverse form has us specifying a string and a namespace, from which we obtain the class data type:

 for item in loop:
     var1 = getattr(rc, str_item)   # Need to remove prefix and get from proper namespace (RC)
     var2 = getattr(rc, str_parent) # May need to do so for parent or item for property
     var1.is_a.append(var2)

The general challenge in this form is to make sure that items and parents are in the form of strings without namespaces, and that the proper namespace is referenced when retrieving the actual attribute value. Many code examples throughout show examples of how to test for trap multiple namespaces.