pyndn.util package

Submodules

pyndn.util.blob module

This module defines the Blob class which holds an immutable byte array. We use an immutable buffer so that it is OK to pass the object into methods because the new or old owner can’t change the bytes. Note that the pointer to the buffer can be None.

class pyndn.util.blob.Blob(array=None, copy=True)

Bases: object

Create a new Blob which holds an immutable array of bytes.

Parameters:

• array (Blob, bytearray, memoryview or other array of int) – (optional) The array of bytes, or None. If array is str (in Python 2) or unicode (in Python 2), then encode using UTF-8. (However, if array is str in Python 2, treat it as a raw string and convert to an array of int.) If you just want a Blob from a raw str without encoding, use Blob.fromRawStr.
• copy (bool) – (optional) If true, copy the contents of array into a new bytearray. If false, just use the existing array without copying. If omitted, then copy the contents (unless value is already a Blob). IMPORTANT: If copy is false, if you keep a pointer to the array then you must treat the array as immutable and promise not to change it.

buf()

Return the byte array which you must treat as immutable and not modify the contents. Note: For compatibility with Python versions before 3.3, if you need an object which implements the buffer protocol (e.g. for writing to a socket) then call toBuffer() instead.

Returns:An array which you should not modify, or None if isNull(). Return type:An array type with int elements, such as bytearray.

compare(other)

Compare this to the other Blob using byte-by-byte comparison. If this and other are both isNull(), then this returns 0. If this isNull() and the other is not, return -1. If this is not isNull() and the other is, return 1. We compare explicitly because a Blob uses a memoryview which doesn’t have built-in comparison operators.

Parameters:other (Blob) – The other Blob to compare with. Returns:0 If they compare equal, -1 if self is less than other, or 1 if self is greater than other. If both are equal up to the shortest, then return -1 if self is shorter than other, or 1 of self is longer than other. Return type:int

equals(other)

Check if this is byte-wise equal to the other Blob. If this and other are both isNull(), then this returns True. We compare explicitly because a Blob uses a memoryview which doesn’t have built-in comparison operators.

Parameters:other (Blob) – The other Blob to compare with. Returns:True if the blobs are equal, otherwise False. Return type:bool

static fromRawStr(rawStr)

Convert rawStr to a Blob. This does not do any character decoding such as UTF-8. If you want decode the string such as UTF-8, then just pass the string to the Blob constructor.

Parameters:rawStr (str) – The raw string to convert to a Blob. Returns:A new Blob created from rawStr. Return type:Blob

isNull()

Return True if the array is None, otherwise False.

Returns:True if the array is None. Return type:bool

size()

Return the length of the immutable byte array.

Returns:The length of the array. Return type:int

toBuffer()

Return an array which implements the buffer protocol (but for Python versions before 3.3 it doesn’t have int elements). This method is only needed by Python versions before 3.3 to check if buf() would return a _memoryviewWrapper and to return its internal memoryview instead. However, if this is a Python version (3.3 or greater) whose memoryview already uses int, then toBuffer() is the same as buf().

Returns:The array which implements the buffer protocol, or None if isNull(). Return type:an array which implements the buffer protocol

toBytes()

Return an object which is the same as the bytes() operator. In Python 2, this simply calls self.toRawStr() because bytes is the same as str. In Python 3, this converts the byte array to a bytes type. This method is necessary because the bytes type is different in Python 2 and 3. This does not do any character encoding such as UTF-8.

Returns:The array as a bytes type, or None if isNull(). Return type:bytes (str in Python 2)

toHex()

Return the hex representation of the bytes in array.

Returns:The hex string. Return type:str

toRawStr()

Return the bytes of the byte array as a raw str of the same length. This does not do any character encoding such as UTF-8.

Returns:The array as a str, or None if isNull(). Return type:str

pyndn.util.boost_info_parser module

class pyndn.util.boost_info_parser.BoostInfoParser

Bases: object

getRoot()

Returns:The root tree of this parser. Return type:BoostInfoTree

read(fileNameOrInput, inputName=None)

Add the contents of the file or input string to the root BoostInfoTree. There are two forms: read(fileName) reads fileName from the file system. read(input, inputName) reads from the input, in which case inputName is used only for log messages, etc.

Parameters:

• fileName (str) – The path to the INFO file.
• input (str) – The contents of the INFO file, with lines separated by NL or CR/NL.
• inputName (str) – Use with input for log messages, etc.

readPropertyList(fromDict)

Import a python dict as a BoostInfoTree. Only leaf nodes will have associated values. :param dict fromDict: The dictionary to import.

write(filename)

Write the root tree of this BoostInfoParser as file in Boost’s INFO format. :param str filename: The output path.

class pyndn.util.boost_info_parser.BoostInfoTree(value=None, parent=None)

Bases: object

addSubtree(treeName, newTree)

Insert a BoostInfoTree as a sub-tree with the given name. :param str treeName: The name of the new sub-tree. :param BoostInfoTree newTree: The sub-tree to add.

clone()

Create a deep copy of this tree.

createSubtree(treeName, value=None)

Create a new BoostInfo and insert it as a sub-tree with the given name. :param str treeName: The name of the new sub-tree. :param str value: The value associated with the new sub-tree. :return: The created sub-tree. :rtype: BoostInfoTree

getValue()

Returns:The value associated with this tree. Return type:str

pyndn.util.boost_info_parser.shlex_split(s)

Similar to shlex.split, split s into an array of strings which are separated by whitespace, treating a string within quotes as a single entity regardless of whitespace between the quotes. Also allow a backslash to escape the next character.

Parameters:s (str) – The input string to split. Returns:An array of strings. Return type:list of str

pyndn.util.change_counter module

This module defines the ChangeCounter class which keeps a target object whose change count is tracked by a local change count. You can set to a new target which updates the local change count, and you can call checkChanged to check if the target (or one of the target’s targets) has been changed. The target object must have a method getChangeCount.

class pyndn.util.change_counter.ChangeCounter(target)

Bases: object

Create a new ChangeCounter to track the given target. If target is not None, this sets the local change counter to target.getChangeCount().

Parameters:target (An type with method getChangeCount()) – The target to track.

checkChanged()

If the target’s change count is different than the local change count, then update the local change count and return True. Otherwise return False, meaning that the target has not changed. Also, if the target is None, simply return false. This is useful since the target (or one of the target’s targets) may be changed and you need to find out.

Returns:True if the change count has been updated, false if not. Return type:bool

get()

Get the target object. If the target is changed, then checkChanged will detect it.

Returns:The target object. Return type:An type with method getChangeCount()

set(target)

Set the target to the given target. If target is not None, this sets the local change counter to target.getChangeCount().

Parameters:target (An type with method getChangeCount()) – The target to track.

pyndn.util.command_interest_generator module

This module defines the CommandInterestGenerator class which keeps track of a timestamp and generates command interests according to the NFD Signed Command Interests protocol: http://redmine.named-data.net/projects/nfd/wiki/Command_Interests

class pyndn.util.command_interest_generator.CommandInterestGenerator

Bases: object

Create a new CommandInterestGenerator and initialize the timestamp to now.

generate(interest, keyChain, certificateName, wireFormat=None)

Append a timestamp component and a random value component to interest’s name. This ensures that the timestamp is greater than the timestamp used in the previous call. Then use keyChain to sign the interest which appends a SignatureInfo component and a component with the signature bits. If the interest lifetime is not set, this sets it.

Parameters:

• interest (Interest) – The interest whose name is append with components.
• keyChain (KeyChain) – The KeyChain for calling sign.
• certificateName (Name) – The certificate name of the key to use for signing.
• wireFormat (A subclass of WireFormat) – (optional) A WireFormat object used to encode the SignatureInfo and to encode interest name for signing. If omitted, use WireFormat.getDefaultWireFormat().

pyndn.util.common module

This module defines the Common class which has static utility functions.

class pyndn.util.common.Common

Bases: object

MAX_NDN_PACKET_SIZE = 8800

static getBytesIOString(bytesIO)

Return bytesIO.getvalue(), making sure the result is a str. This is necessary because getvalue() returns a bytes object in Python 3.

static getNowMilliseconds()

Get the current time in milliseconds.

Returns:The current time in milliseconds since 1/1/1970, including fractions of a millisecond. Return type:float

static stringToUtf8Array(input)

If the input has type str (in Python 3) or unicode (in Python 2), then encode it as UTF8 and return an array of integers. If the input is str (in Python 2) then treat it as a “raw” string and just convert each element to int. Otherwise, if the input is not str or unicode, just return the input. This is necessary because in Python 3 doesn’t have the unicode type and the elements in a string a Unicode characters. But in Python 2 only the unicode type has Unicode characters, and str elements are bytes with value 0 to 255 (and often used to carry binary data).

static typeIsString(obj)

Check if obj has type str or (in Python 2) unicode. This is necessary because Python 2 has two string types, str and unicode, but Python 3 doesn’t have type unicode so we have to be carefor to check for type(obj) is unicode.

Parameters:obj (any) – The object to check if it is str or unicode. Returns:True if obj is str or unicode, otherwise false Return type:bool

pyndn.util.dynamic_byte_array module

This module defines the DynamicByteArray class which holds a bytearray which can be expanded as needed.

class pyndn.util.dynamic_byte_array.DynamicByteArray(length=16)

Bases: object

Create a new DynamicByteArray with an initial bytearray.

Parameters:length (int) – (optional) The initial length of the bytearray. If omitted, use a default value.

copy(value, offset)

First call ensureLength to make sure the bytearray has offset + len(value) bytes, then copy value into the bytearray starting at offset.

Parameters:

• value (bytearray or memoryview) – The byte array with the bytes to copy.
• offset (int) – The offset in the array to start copying into.

copyFromBack(value, offsetFromBack)

First call ensureLengthFromBack to make sure the bytearray has offsetFromBack bytes, then copy value into the bytearray starting offsetFromBack bytes from the back of the array.

Parameters:

• value (bytearray or memoryview) – The byte array with the bytes to copy.
• offsetFromBack (int) – The offset from the back of the array to start copying.

ensureLength(length)

Ensure length. If necessary, reallocate the bytearray and copy existing data to the front of the new array.

Parameters:length (int) – The minimum length for the bytearray.

ensureLengthFromBack(length)

Ensure length. If necessary, reallocate the bytearray and shift existing data to the back of the new array.

Parameters:length (int) – The minimum length for the bytearray.

getArray()

Get the bytearray. After more method calls, the result of getArray() may change.

Returns:The bytearray. Return type:bytearray.

pyndn.util.exponential_re_express module

This module defines the ExponentialReExpress class which uses an internal onTimeout to express the interest again with double the interestLifetime. See ExponentialReExpress.makeOnTimeout, which you should call instead of the private constructor. Create a new ExponentialReExpress where onTimeout expresses the interest again with double the interestLifetime. If the interesLifetime goes over settings.maxInterestLifetime, then call the given onTimeout. If this internally gets onData, just call the given onData.

class pyndn.util.exponential_re_express.ExponentialReExpress(face, onData, onTimeout, maxInterestLifetime)

Bases: object

static makeOnTimeout(face, onData, onTimeout, maxInterestLifetime=None)

Return a function object to use in expressInterest for onTimeout which will express the interest again with double the interestLifetime. If the interesLifetime goes over maxInterestLifetime (see below), then call the provided onTimeout. If a Data packet is received, this calls the provided onData. Use it like this: def onData: ... def onTimeout ... face.expressInterest(interest, onData, ExponentialReExpress.makeOnTimeout(face, onData, onTimeout))

Parameters:

• face (Face) – This calls face.expressInterest.
• onData (function object) – When a matching data packet is received, this calls onData(interest, data) where interest is the interest given to expressInterest and data is the received Data object. This is normally the same onData you initially passed to expressInterest. NOTE: The library will log any exceptions thrown by this callback, but for better error handling the callback should catch and properly handle any exceptions.
• onTimeout (function object) – If the interesLifetime goes over maxInterestLifetime, this calls onTimeout(interest). However, if onTimeout is None, this does not use it. NOTE: The library will log any exceptions thrown by this callback, but for better error handling the callback should catch and properly handle any exceptions.
• maxInterestLifetime (float) – (optional) The maximum lifetime in milliseconds for re-expressed interests. If omitted, use 16000.

pyndn.util.memory_content_cache module

This module defines the MemoryContentCache class which holds a set of Data packets and answers an Interest to return the correct Data packet. The cache is periodically cleaned up to remove each stale Data packet based on its FreshnessPeriod (if it has one). Note: This class is an experimental feature. See the API docs for more detail at http://named-data.net/doc/ndn-ccl-api/memory-content-cache.html .

class pyndn.util.memory_content_cache.MemoryContentCache(face, cleanupIntervalMilliseconds=None)

Bases: object

Create a new MemoryContentCache to use the given Face.

Parameters:

• face (Face) – The Face to use to call registerPrefix and setInterestFilter, and which will call this object’s OnInterest callback.
• cleanupIntervalMilliseconds (float) – (optional) The interval in milliseconds between each check to clean up stale content in the cache. If omitted, use a default of 1000 milliseconds. If this is a large number, then effectively the stale content will not be removed from the cache.

add(data)

Add the Data packet to the cache so that it is available to use to answer interests. If data.getMetaInfo().getFreshnessPeriod() is not None, set the staleness time to now plus data.getMetaInfo().getFreshnessPeriod(), which is checked during cleanup to remove stale content. This also checks if cleanupIntervalMilliseconds milliseconds have passed and removes stale content from the cache. After removing stale content, remove timed-out pending interests from storePendingInterest(), then if the added Data packet satisfies any interest, send it through the face and remove the interest from the pending interest table.

Parameters:data (Data) – The Data packet object to put in the cache. This copies the fields from the object.

getStorePendingInterest()

Return a callback to use for onDataNotFound in registerPrefix which simply calls storePendingInterest() to store the interest that doesn’t match a Data packet. add(data) will check if the added Data packet satisfies any pending interest and send it.

Returns:A callback to use for onDataNotFound in registerPrefix(). Return type:function object

registerPrefix(prefix, onRegisterFailed, onRegisterSuccess=None, onDataNotFound=None, flags=None, wireFormat=None)

Call registerPrefix on the Face given to the constructor so that this MemoryContentCache will answer interests whose name has the prefix. Alternatively, if the Face’s registerPrefix has already been called, then you can call this object’s setInterestFilter.

Parameters:

• prefix (Name) – The Name for the prefix to register. This copies the Name.
• onRegisterFailed (function object) – If this fails to register the prefix for any reason, this calls onRegisterFailed(prefix) where prefix is the prefix given to registerPrefix. NOTE: The library will log any exceptions raised by this callback, but for better error handling the callback should catch and properly handle any exceptions.
• onRegisterSuccess (list of one function object) – (optional) This calls onRegisterSuccess[0](prefix, registeredPrefixId) when this receives a success message from the forwarder. If onRegisterSuccess is omitted or [None], this does not use it. (As a special case, this optional parameter is supplied as a list of one function object, instead of just a function object, in order to detect when it is used instead of the following optional onDataNotFound function object.) NOTE: The library will log any exceptions raised by this callback, but for better error handling the callback should catch and properly handle any exceptions.
• onDataNotFound (function object) – (optional) If a data packet for an interest is not found in the cache, this forwards the interest by calling onDataNotFound(prefix, interest, face, interestFilterId, filter). Your callback can find the Data packet for the interest and call face.putData(data). If your callback cannot find the Data packet, it can optionally call storePendingInterest(interest, face) to store the pending interest in this object to be satisfied by a later call to add(data). If you want to automatically store all pending interests, you can simply use getStorePendingInterest() for onDataNotFound. If onDataNotFound is omitted or None, this does not use it. NOTE: The library will log any exceptions raised by this callback, but for better error handling the callback should catch and properly handle any exceptions.
• flags (ForwardingFlags) – (optional) See Face.registerPrefix.
• wireFormat (A subclass of WireFormat) – (optional) See Face.registerPrefix.

setInterestFilter(filterOrPrefix, onDataNotFound=None)

Call setInterestFilter on the Face given to the constructor so that this MemoryContentCache will answer interests whose name matches the filter. There are two forms of setInterestFilter. The first form uses the exact given InterestFilter: setInterestFilter(filter, [onDataNotFound]). The second form creates an InterestFilter from the given prefix Name: setInterestFilter(prefix, [onDataNotFound]).

Parameters:

• filter (InterestFilter) – The InterestFilter with a prefix and optional regex filter used to match the name of an incoming Interest. This makes a copy of filter.
• prefix (Name) – The Name prefix used to match the name of an incoming Interest. This makes a copy of the Name.
• onDataNotFound (function object) – (optional) If a data packet for an interest is not found in the cache, this forwards the interest by calling onDataNotFound(prefix, interest, face, interestFilterId, filter). Your callback can find the Data packet for the interest and call face.putData(data). If your callback cannot find the Data packet, it can optionally call storePendingInterest(interest, face) to store the pending interest in this object to be satisfied by a later call to add(data). If you want to automatically store all pending interests, you can simply use getStorePendingInterest() for onDataNotFound. If onDataNotFound is omitted or None, this does not use it. NOTE: The library will log any exceptions raised by this callback, but for better error handling the callback should catch and properly handle any exceptions.

storePendingInterest(interest, face)

Store an interest from an OnInterest callback in the internal pending interest table (normally because there is no Data packet available yet to satisfy the interest). add(data) will check if the added Data packet satisfies any pending interest and send it through the face.

Parameters:

• interest (Interest) – The Interest for which we don’t have a Data packet yet. You should not modify the interest after calling this.
• face (Face) – The Face with the connection which received the interest. This comes from the OnInterest callback.

unregisterAll()

Call Face.unsetInterestFilter and Face.removeRegisteredPrefix for all the prefixes given to the setInterestFilter and registerPrefix method on this MemoryContentCache object so that it will not receive interests any more. You can call this if you want to “shut down” this MemoryContentCache while your application is still running.

pyndn.util.ndn_regex module

class pyndn.util.ndn_regex.NdnRegexMatcher

Bases: object

static match(pattern, name)

Determine if the provided NDN regex matches the given Name. :param str pattern: The NDN regex. :param Name name: The Name to match against the regex.

pyndn.util.segment_fetcher module

This module defines the SegmentFetcher class which is a utility class to fetch the latest version of segmented data.

SegmentFetcher assumes that the data is named /<prefix>/<version>/<segment>, where:

• <prefix> is the specified name prefix,
• <version> is an unknown version that needs to be discovered, and
• <segment> is a segment number. (The number of segments is unknown and is controlled by the FinalBlockId field in at least the last Data packet.

The following logic is implemented in SegmentFetcher:

1. Express the first Interest to discover the version:

   >> Interest: /<prefix>?ChildSelector=1&MustBeFresh=true

2. Infer the latest version of the Data: <version> = Data.getName().get(-2)

3. If the segment number in the retrieved packet == 0, go to step 5.

4. Send an Interest for segment 0:

   >> Interest: /<prefix>/<version>/<segment=0>

5. Keep sending Interests for the next segment while the retrieved Data does not have a FinalBlockId or the FinalBlockId != Data.getName().get(-1).

   >> Interest: /<prefix>/<version>/<segment=(N+1))>

6. Call the onComplete callback with a Blob that concatenates the content from all the segmented objects.

If an error occurs during the fetching process, the onError callback is called with a proper error code. The following errors are possible:

• INTEREST_TIMEOUT: if any of the Interests times out
• DATA_HAS_NO_SEGMENT: if any of the retrieved Data packets don’t have a segment as the last component of the name (not counting the implicit digest)
• SEGMENT_VERIFICATION_FAILED: if any retrieved segment fails the user-provided VerifySegment callback

In order to validate individual segments, a verifySegment callback needs to be specified. If the callback returns False, the fetching process is aborted with SEGMENT_VERIFICATION_FAILED. If data validation is not required, the provided DontVerifySegment object can be used.

Example:

def onComplete(content):

...

def onError(errorCode, message):

...

interest = Interest(Name(“/data/prefix”)) interest.setInterestLifetimeMilliseconds(1000)

SegmentFetcher.fetch(

face, interest, SegmentFetcher.DontVerifySegment, onComplete, onError)

class pyndn.util.segment_fetcher.SegmentFetcher(face, verifySegment, onComplete, onError)

Bases: object

A private constructor to create a new SegmentFetcher to use the Face. An application should use SegmentFetcher.fetch.

Parameters:

• face (Face) – This calls face.expressInterest to fetch more segments.
• verifySegment (function object) – When a Data packet is received this calls verifySegment(data) where data is a Data object. If it returns False then abort fetching and call onError with SegmentFetcher.ErrorCode.SEGMENT_VERIFICATION_FAILED.
• onComplete (function object) – When all segments are received, call onComplete(content) where content is a Blob which has the concatenation of the content of all the segments. NOTE: The library will log any exceptions raised by this callback, but for better error handling the callback should catch and properly handle any exceptions.
• onError (function object) – Call onError.onError(errorCode, message) for timeout or an error processing segments. errorCode is a value from SegmentFetcher.ErrorCode and message is a related string. NOTE: The library will log any exceptions raised by this callback, but for better error handling the callback should catch and properly handle any exceptions.

static DontVerifySegment(data)

DontVerifySegment may be used in fetch to skip validation of Data packets.

class ErrorCode

Bases: object

An ErrorCode value is passed in the onError callback.

DATA_HAS_NO_SEGMENT = 2

INTEREST_TIMEOUT = 1

SEGMENT_VERIFICATION_FAILED = 3

static SegmentFetcher.fetch(face, baseInterest, verifySegment, onComplete, onError)

Initiate segment fetching. For more details, see the documentation for the module.

Parameters:

• face (Face) – This calls face.expressInterest to fetch more segments.
• baseInterest (Interest) – An Interest for the initial segment of the requested data, where baseInterest.getName() has the name prefix. This interest may include a custom InterestLifetime and selectors that will propagate to all subsequent Interests. The only exception is that the initial Interest will be forced to include selectors “ChildSelector=1” and “MustBeFresh=true” which will be turned off in subsequent Interests.
• verifySegment (function object) – When a Data packet is received this calls verifySegment(data) where data is a Data object. If it returns False then abort fetching and call onError with SegmentFetcher.ErrorCode.SEGMENT_VERIFICATION_FAILED. If data validation is not required, use SegmentFetcher.DontVerifySegment.
• onComplete (function object) – When all segments are received, call onComplete(content) where content is a Blob which has the concatenation of the content of all the segments. NOTE: The library will log any exceptions raised by this callback, but for better error handling the callback should catch and properly handle any exceptions.
• onError (function object) – Call onError.onError(errorCode, message) for timeout or an error processing segments. errorCode is a value from SegmentFetcher.ErrorCode and message is a related string. NOTE: The library will log any exceptions raised by this callback, but for better error handling the callback should catch and properly handle any exceptions.

pyndn.util.signed_blob module

This module defines the SignedBlob class which extends Blob to keep the offsets of a signed portion (e.g., the bytes of Data packet).

class pyndn.util.signed_blob.SignedBlob(blob=None, signedPortionBeginOffset=None, signedPortionEndOffset=None)

Bases: pyndn.util.blob.Blob

Create a new SignedBlob using the given Blob and offsets.

Parameters:

• blob (Blob or SignedBlob) – (optional) The Blob with a signed portion. If omitted, then isNull() is True.
• signedPortionBeginOffset (int) – (optional) The offset in the buffer of the beginning of the signed portion.
• signedPortionEndOffset (int) – (optional) The offset in the buffer of the end of the signed portion.

signedBuf()

Return the signed portion of the byte array which you must treat as immutable and not modify the contents.

Returns:An array which you should not modify, or None if isNull(). Return type:An array type with int elements, such as bytearray.

signedSize()

Get the length of the signed portion of the immutable byte buffer.

Returns:The length of the signed portion, or 0 if isNull(). Return type:int

toSignedBuffer()

Return an array of the signed portion which implements the buffer protocol (but for Python versions before 3.3 it doesn’t have int elements). This method is only needed by Python versions before 3.3 to check if signedBuf() would return a _memoryviewWrapper and to return its internal memoryview instead. However, if this is a Python version (3.3 or greater) whose memoryview already uses int, then toSignedBuffer() is the same as signedBuf().

toSignedBytes()

Return an object which is the same as the bytes() operator of the signed portion. In Python 2, this makes a raw string because bytes is the same as str. In Python 3, this converts the byte array to a bytes type. This method is necessary because the bytes type is different in Python 2 and 3. This does not do any character encoding such as UTF-8.

Returns:The array as a bytes type, or None if isNull(). Return type:bytes (str in Python 2)

Module contents