npu.build package

Submodules

npu.build.appbuilder module

class npu.build.appbuilder.AppBuilder(name=None)

Bases: object

This class is able to build NPU applications from a high level description that is specified in the callgraph() method using npu.build objects.

name

The name of the application.

Type:

str

ab

The xclbin builder class used to build the final xclbin binary.

Type:

AppXclbinBuilder

fxtracer

The apptracer class used to build the application metadata.

Type:

AppTracer

kernels

Dictionary storing all unique compute tile kernels in this application.

Type:

dict

connections

Dictionary storing all unique connections between kernels in this application.

Type:

dict

previous_build_args

List containing the input arguments last used to build the application.

Type:

list

Note

This class is typically meant to be subclassed with a custom callgraph(). Many examples of this subclassing pattern are in teststest_applications.py.

build(*args, debug=False, mlir=None)

The application is built using the callgraph call with supplied arguments.

callgraph()

This method should be overridden by a subclass.

display()

Generates the application SVG and displays inside a IPython environment.

Return type:

None

displaymlir(*args, what='')

Displays part of the application in a IPython friendly way.

Return type:

None

merge_applications(newkernels, newconnections)

property metadata

Generates the application JSON and displays inside a IPython environment.

save(filename=None)

saves animation to a file.

Return type:

None

to_handoff(*args, file=None)

Converts the application into a serializable JSON file.

to_json(*args)

Converts the application into JSON.

to_metadata(*args)

The application is converted into the AppMetadata after tracing the callgraph() call.

to_mlir(*args, file=None)

Generates the application mlir file from the metadata collected.

to_sequence()

Generates the application data movement sequence from the connections traced.

unique_named(objs)

validate_previous_build_args()

npu.build.appmetadata module

class npu.build.appmetadata.AppMetada(appname, kernels, connections, sequence)

Bases: object

This class contains the in-memory representation of the AppBuilder application and completes any unspecified application placements (E.g. ComputeTile kernel mapping).

appname

The name of the application.

Type:

str

kernels

Dictionary storing all unique compute tile kernels in this application.

Type:

dict

connections

Dictionary storing all unique connections between kernels in this application.

Type:

dict

sequence

List containing the ordered data movements as traced in the AppBuilder’s callgraph.

Type:

list

to_json()

npu.build.apptracer module

class npu.build.apptracer.AppTracer(application)

Bases: object

This class manages the tracing of an AppBuilder’s callgraph method to determine: 1. unique kernels used in the callgraph 2. unique connections between IT, MT, and CT kernels 3. ordered sequence of data movements between kernels using connections

application

The AppBuilder instance passed in with a valid callgraph for tracing

Type:

AppBuilder

postprocess_args(kernel, args)

Postprocess args to get buffer ports, RTP values and any additional kernels that are passed into functions directly.

postprocess_kernel(k, res)

Kernel type specific postprocessing to get the right object for tracing.

to_trace(*args)

Using the TraceLogger class, the application’s callgraph is traced and returns discovered kernels and connections.

npu.build.appxclbinbuilder module

class npu.build.appxclbinbuilder.AppXclbinBuilder

Bases: WSLBuilder

This class manages the WSL building of the application, resulting in

the final xclbin binary that is run on the NPU.

xclbin

Path to the NPU executable xclbin file.

Type:

str

buildlog

Path to the WSL build logfile.

Type:

str

build(appname, mlir_file, kernels, debug=False)

Toplevel call to build the xclbin

Parameters:

• appname (string) – The desired final name of the xclbin.

• mlir_file (string) – The filepath containing the MLIR to compile.

• kernels (list) – The list of kernels that need to be compiled first before building the xclbin.

• debug (boolean) – Optional setting to deliver extra logging and debug messages from the build.

Return type:

None

build_kernels(kernels, debug)

Build the ComputeTile kernel object files if not done already.

npu.build.buffers module

class npu.build.buffers.Buffer(array, slices, name, buffertiletype, disable_unique_name_id=False)

Bases: KernelMeta

This class is a superclass used for IT Buffers and MT Buffer objects. The class

contains a numpy array and any slices used to index into that array. These slices need tracked as data moving to/from IT and MT buffers will result in DMA buffer descriptors based on those slices.

_array

The underlying buffer.

Type:

ndarray

slices

TAn ordered list of slices to index into the underlying array.

Type:

list

property array

return the array after applying the tracked slices.

createports(num_inputs, num_outputs, port_slices=None)

Create a set of bufferports for this Buffer object.

property dtype

property nbytes

property shape

to_metadata()

Produces a dict of the metadata for the kernel

classmethod to_ndarray(a)

class npu.build.buffers.ITBuffer(array, slices=None, setitem_itkernel=None)

Bases: Buffer

An Interface Tile Buffer subclass from Buffer.

classmethod ndarray_to_itb_port(a, setitem_itkernel=None)

classmethod ndarray_to_itbuffer(a, setitem_itkernel=None)

class npu.build.buffers.MTBuffer(array, slices=None, reuse_buffername=None, mtmode='passthrough')

Bases: Buffer

A Memory Tile Buffer subclass from Buffer.

to_metadata()

Produces a dict of the metadata for the kernel

npu.build.connections module

class npu.build.connections.Connection(src_sink_tuple)

Bases: object

This class holds the source and sink objects for a connection within

an NPU application. Sources and sinks can be IT, MT, or CTs and the name of a connection can be split up to obtain dictionary keys for the src, src_port, snk and snk_port which is then used in AppBuilder tracing.

Notes

Connections are point to point links. Broadcast links are inferred by analyzing multiple Connection objects. Connection names are built as srcname__srcportname___snkname___snkportname.

srckernel

The source kernel (IT, MT, or CT) for this connection.

Type:

str

srcport

The source port for this connection.

Type:

str

sinkkernel

The sink kernel (IT, MT, or CT) for this connection.

Type:

str

sinkport

The sink port for this connection.

Type:

str

name

Name of this connection constructed from snk/src names.

Type:

str

dtype

Type information for the connection.

Type:

str

ctype

The implementation type of this connection, E.g. ‘rtp’, ‘objfifo,pingpong’.

Type:

str

npu.build.itkernel module

class npu.build.itkernel.ITKernel(fxname)

Bases: KernelMeta

This class is a superclass for all Kernels that run against Interface Tile buffers.

These kernels can take an optional bufref to reuse an existing IT Buffer or by default create a new IT buffer when called.

itbuf

The Interface Tile Buffer that is used within a ITKernel.

Type:

BufferPort

class npu.build.itkernel.ITRead(inputbuffer=None)

Bases: object

This class is used to implement an IT Read Call.

inputbuffer

The buffer that is to be read.

Type:

BufferPort

class npu.build.itkernel.ITReadCall

Bases: ITKernel

This class is used to execute the IT Read kernel.

class npu.build.itkernel.ITWrite(inputbuffer=None, bufref=None)

Bases: object

This class is used to wrap a ITWriteCall object using __new__.

inputbuffer

The buffer to be written into an IT Buffer.

Type:

BufferPort

buferf

Optional existing IT Buffer that is used for storing the incoming write.

Type:

BufferPort

class npu.build.itkernel.ITWriteCall

Bases: ITKernel

This class is used to execute the IT Write kernel.

npu.build.kernel module

class npu.build.kernel.Kernel(srccode, behavioralfx=None, top_function=None, requires_boilerplate=False)

Bases: KernelMeta

This class encapsulates a ComputeTile kernel C/C++ src code and methods to generate a compiled object - that compiled object is used within MLIR to build the final xclbin application. Additionally, the kernel is parsed for input and output ports and can encapsulates a behavioral model to capture functional behavior and data shaping from input to output ports. This metadata for the kernel enables behavioral execution to verify correctness in Python and also tracing to build the final AppBuilder xclbin.

srccode

The C/C++ source code of the ComputeTile kernel.

Type:

str

srcfile

The C/C++ source file path.

Type:

str

kname

The name of this kernel instance.

Type:

str

behavioralfx

The behavioral function that emulates the C/C++ kernel’s behavior.

Type:

function

build(debug=False)

Build the kernel object file for linking into the complete application.

completed_srccode()

From the parsed information generate the source.

Return type:

str

create_outputs(behavioral_n_tracing)

From kernel call, produce the output value or tuple.

display()

Render the kernel code in a jupyter notebook.

Return type:

None

property objfile

to_cpp()

output source code to a .cpp file

Return type:

None

npu.build.kernelbuilder module

class npu.build.kernelbuilder.KernelObjectBuilder(name, srccode, srcfile)

Bases: WSLBuilder

This class builds ComputeTile kernel C/C++ into object files for linking into applications.

There is also caching support so that a kernel is only built one-time.

name

The name of the kernel.

Type:

str

srccode

The C/C++ source code of the ComputeTile kernel.

Type:

str

srcfile

The C/C++ source file path.

Type:

str

Notes

The cache of object files already built can be cleared by running KernelObjectBuilder.clear_cache().

build(debug=False)

Build the kernel object file and copy it to self.prebuilt_objpath.

cached_objfile_exists()

Check if cached object file exists built from identical source code.

classmethod clear_cache()

prebuilt_path = 'C:\\Users\\mruiznog\\AppData\\Local\\Riallto\\riallto_venv\\lib\\site-packages\\npu\\build\\..\\lib\\cached'

update_cache_md5()

npu.build.kernelmeta module

class npu.build.kernelmeta.KernelMeta(name, shortname, ktype, ttype, ports=None, tloc=None, disable_unique_name_id=False)

Bases: object

This class is a superclass for all Kernels in this package - including IT, MT, and CT kernels. it keeps track of unique names for kernels as they are instantiated and has helper functions to identify and iterate over kernel ports.

name

The name of this kernel.

Type:

str

shortname

an abbreviated name for use in application metadata.

Type:

str

ktype

The kernel type - will be defined by the operation performed (E.g. ‘mtpassthrough’ for a MemoryTile passthrough operation).

Type:

str

ttype

The kernel tile type - valid values are ‘IT’, ‘MT’ or ‘CT’.

Type:

str

ports

The kernel ports which are defined by the kernel operation.

Type:

list

tloc

The requested Tile location for this kernel as an (x,y) pair.

Type:

tuple

disable_unique_name_id

Optional flag for disabling a unique name for this kernel. If set, the name input argument will be used.

Type:

boolean

property bufferports

property inputbufferports

property metadata

property outputbufferports

classmethod reset_unique_names()

property rtpports

to_metadata()

Produces a dict of the metadata for the kernel

classmethod unique_name(kernelname)

used_names = {}

npu.build.mlirbuilder module

class npu.build.mlirbuilder.MLIRBuilder(metadata, config=(4, 1, 1))

Bases: object

This class builds an MLIR representation of an application starting from AppMetadata.

metadata

The application metadata.

Type:

AppMetadata

app

The json representation of an application metadata.

Type:

JSON

kernels

Dictionary storing all unique compute tile kernels in this application.

Type:

dict

connections

Dictionary storing all unique connections between kernels in this application.

Type:

dict

sequence

List of the application data movements between kernels.

Type:

list

config

Tuple containing number of CT, MT, IT tiles that can be used to build the MLIR.

Type:

tuple

to_mlir(file=None)

Toplevel method to generate the application MLIR.

npu.build.mlirconnections module

class npu.build.mlirconnections.MLIRConnect(name, src, dsts, nbytes, offset, apptiles, appkernels)

Bases: object

This class represents a connection in MLIR between kernels. Broadcast connections are supported.

name

MLIR variable name of this connection.

Type:

str

id

Incrementing unique id for this connection.

Type:

int

src

The source kernel and port.

Type:

tuple

dsts

List of destination sink kernel and ports.

Type:

list

nbytes

Number of bytes transferred on this connection.

Type:

int

offset

Buffer offset for the source data transfer.

Type:

int

config

Tuple containing number of CT, MT, IT tiles that can be used to build the MLIR.

Type:

tuple

classmethod next_id()

classmethod reset_id()

unique_id = -1

class npu.build.mlirconnections.ObjectFIFO(name, src, dsts, nbytes, offset, apptiles, appkernels)

Bases: MLIRConnect

This class represents an object fifo connection in MLIR between kernels.

elementname

MLIR objectfifo element name.

Type:

str

subviewname

MLIR objectfifo subview name.

Type:

str

to_mlir_acquire(io, indent)

Return the MLIR text for objectfifo acquire calls.

to_mlir_declare()

Return the MLIR text for declaring this object

to_mlir_release(io, indent)

Return the MLIR text for objectfifo release calls.

npu.build.mlirsequencebuilder module

class npu.build.mlirsequencebuilder.MLIRSequnceBuilder(app_metadata, aietiles, cons_broadcasts)

Bases: object

This class generates the MLIR Sequence dialect that describes datamovement to and from The NPU. This is accomplished by analyzing the datamovement for IT Buffers discovered by running the AppBuilder callgraph. The required sync signals are also generated.

_metadata

The application metadata.

Type:

AppMetadata

aietiles

List of AIE tiles in this application.

Type:

list

_userbuffers

Dictionary of IT Buffers discovered during application tracing.

Type:

dict

_ubname2externid

Dictionary of unique incrementing IDs for IT Buffers.

Type:

dict

_ingress_ub

Dictionary of incoming IT Buffers to the NPU array.

Type:

dict

_egress_ub

Dictionary of outgoing IT Buffers from the NPU array.

Type:

dict

_ingress_egress_ub

Dictionary of IT Buffers if the buffer is both input and output to the NPU array.

Type:

dict

_constants_table

Dictionary of constants used in the MLIR sequence specification.

Type:

dict

_cons_broadcasts

List of broadcast connection names.

Type:

list

property mlir: str

Generates the MLIR sequence dialect from the callgraph traced sequence.

class npu.build.mlirsequencebuilder.UBDataMovement(ubname, symname, shape, dtype, tilesizes=<factory>, dim=<factory>, srcoffsets=<factory>, snkoffsets=<factory>)

Bases: object

dim: List[int]

dtype: str

shape: tuple

snkoffsets: List[int]

srcoffsets: List[int]

symname: str

tilesizes: List[int]

ubname: str

npu.build.mlirtiles module

class npu.build.mlirtiles.BufferTile(x, y, buffers=None)

Bases: Tile

Class to represent IT and MT Tiles in MLIR.

is_used()

class npu.build.mlirtiles.CTTile(x, y, kernel=None)

Bases: Tile

This class represents a CT Tile in MLIR.

tloc

The x,y coordinates of this tile.

Type:

tuple

kernel

Dictionary describing the kernel placed on this tile.

Type:

dict

objfifos_produce

list of objectfifos used as a producer by this tile.

Type:

list

objfifos_consume

list of objectfifos used as a consumer by this tile.

Type:

list

is_used()

property objfifos

to_mlir(indent='   ')

Return the MLIR string to represent this tile.

to_mlir_kernel_declare()

to_mlir_tile_declare(indent='')

class npu.build.mlirtiles.ITTile(x, y, buffers=None)

Bases: BufferTile

Class to represent IT Tiles in MLIR.

class npu.build.mlirtiles.MEMTile(x, y, buffers=None)

Bases: BufferTile

Class to represent MT Tiles in MLIR.

class npu.build.mlirtiles.Tile(x, y)

Bases: object

This class represents a Tile in MLIR. IT, MT and CT Tiles are supported.

tloc

The x,y coordinates of this tile.

Type:

tuple

tvarname

The MLIR variable name for this tile.

Type:

str

to_mlir_tile_declare(indent='')

npu.build.mtkernel module

class npu.build.mtkernel.MTConcat(inputbuffers=None)

Bases: object

This class is used to wrap a MTConcatCall object using __new__.

inputbuffers

The list of bufferPorts to be concatenated.

Type:

List

buferf

Optional existing MT Buffer that is used for storing the incoming BufferPorts.

Type:

BufferPort

class npu.build.mtkernel.MTConcatCall

Bases: MTKernel

This class is used to execute the MT Concat kernel.

class npu.build.mtkernel.MTKernel(fxname)

Bases: KernelMeta

This class is a superclass for all MemoryTile Kernels that run against Memory Tile buffers.

These kernels can take an optional bufref to reuse an existing MT Buffer or by default create a new MT buffer when called.

mtbuf

The Memory Tile Buffer that is used within a MTKernel.

Type:

BufferPort

class npu.build.mtkernel.MTPassThrough(inputbuffer=None)

Bases: object

This class is used to wrap a MTPassThroughCall object using __new__.

inputbuffer

The buffer to be written into an MT Buffer.

Type:

BufferPort

buferf

Optional existing MT Buffer that is used for storing the incoming write.

Type:

BufferPort

class npu.build.mtkernel.MTPassThroughCall

Bases: MTKernel

This class is used to execute the MT PassThrough kernel.

class npu.build.mtkernel.MTSplit(*args)

Bases: object

This class is used to wrap a MTSplitCall object using __new__.

inputbuffer

The buffer to be written into an MT Buffer.

Type:

BufferPort

numsplits

The number of buffer splits requested to be returned.

Type:

int

buferf

Optional existing MT Buffer that is used for storing the incoming write.

Type:

BufferPort

class npu.build.mtkernel.MTSplitCall(numsplits)

Bases: MTKernel

This class is used to execute the MT Split kernel.

npu.build.port module

class npu.build.port.BufferPort(name, pdtype=None, array=None, slices=None, io=None, ctype=None, parent=None)

Bases: Port

This class represents a kernel port that contains a buffer. That buffer is held as a NumPy array and can be sliced to collect an offset into the array if DMA transfers are used to move the buffer. All IT and MT ports are bufferports, whereas CT kernels can have a mix of RTP and BufferPorts.

_array

The underlying buffer.

Type:

ndarray

slices

TAn ordered list of slices to index into the underlying array.

Type:

list

property array

Returns the array after all slices have been applied.

copy(newparent=None, newslices=None)

property nbytes

property offset

Return the byte offset into the original array after slices are applied.

property shape

to_metadata()

produce a dict of metadata for the port

class npu.build.port.Port(name, pdtype=None, io=None, ctype=None, parent=None, c_dtype=None)

Bases: object

This class holds the attributes for a Kernel port in an AppBuilder application.

name

Name of this connection constructed from snk/src names.

Type:

str

pdtype

The port datatype.

Type:

str

io

The port direction, either ‘in’ or ‘out’.

Type:

str

ctype

The connection type of this port.

Type:

str

parent

The Kernel object that contains this Port.

Type:

Kernel

c_dtype

Type information for the connection.

Type:

str

property metadata

to_metadata()

produce a dict of metadata for the port

class npu.build.port.RTPPort(name, value=None, parent=None, c_dtype=None)

Bases: Port

This class represents a Run Time Parameter (RTP) port to a Kernel.

value

the RTP’s value that is updated by userspace writes.

Type:

int

copy(newparent=None)

to_metadata()

produce a dict of metadata for the port

npu.build.sequence module

class npu.build.sequence.SequenceList(connections)

Bases: object

This class hold sequences as an ordered list and produces the metadata for each sequence item.

to_seqitems()

npu.build.tracekernels module

npu.build.tracekernels.isfunctraceable(func)

npu.build.tracekernels.kerneltracer(func)

Wrapper around traced functions to enable logging of calls in the global TraceLogger.

npu.build.tracelogger module

class npu.build.tracelogger.TraceLogger

Bases: object

Global class used for tracing an AppBuilder callgraph.

classmethod clear_trace_objects()

is_traceenabled = False

trace_kernelcalls = []

classmethod update_trace_objects(newkernelcall)

npu.build.userspace module

class npu.build.userspace.UserspaceRTP(value)

Bases: KernelMeta

This class contains a userspace port for acessing NPU runtime paraemters (RTP).

npu.build.utils module

npu.build.utils.check_wsl_install()

Return type:

None

npu.build.utils.is_win()

Returns true if we are running this on Windows.

Return type:

bool

npu.build.utils.is_win_path(path)

Returns true if the path above is a Windows path

Return type:

bool

npu.build.utils.is_wsl_win_path(path)

Returns true if this is a windows path into WSL

Return type:

bool

npu.build.utils.wsl_prefix()

if we are running this on windows return the appropriate wsl prefix.

Return type:

str

npu.build.wslbuilder module

class npu.build.wslbuilder.WSLBuilder

Bases: object

This class contains methods to call into WSL to build NPU CT kernels and applications.

_tmp_dir

The temporary directory where builds will occur.

Type:

TemporaryDirectory

Module contents

npu.build.wslpath(winpath)

From the windows path create the equivalent WSL path

Return type:

str