NVM Express Overview

NVM Express is a scalable host controller interface designed to address the needs of Enterprise, Data Center and Client systems that utilize PCI Express® (PCIe®) based solid state drives. The interface provides an optimized command issue and completion path. It includes support for parallel operation by supporting up to 64K commands within a single I/O queue to the device. Additionally, support has been added for many Enterprise capabilities like end-to-end data protection (compatible with T10 DIF and DIX standards), enhanced error reporting, and virtualization.

NVM Express is based on a paired Submission and Completion Queue mechanism. Commands are placed by host software into the Submission Queue. Completions are placed into an associated Completion Queue by the controller. Multiple Submission Queues may utilize the same Completion Queue. The Submission and Completion Queues are allocated in host memory.

NVMe Controller

An Admin Submission and associated Completion Queue exist for the purpose of device management and control – e.g., creation and deletion of I/O Submission and Completion Queues, aborting commands, etc. Only commands that are part of the Admin Command Set may be issued to the Admin Submission Queue.

Host software creates queues, up to the maximum supported by the controller. Typically the number of command queues created is based on the system configuration and anticipated workload. For example, on a four core processor based system, there may be a queue pair per core to avoid locking and ensure data structures are created in the appropriate processor core’s cache.

Utilizing this framework, the result is efficient and scalable performance with low latency.

NVM Express Key attributes:

  • Does not require un-cacheable / MMIO register reads in the command issue or completion path.
  • A maximum of one MMIO register write is necessary in the command issue path.
  • Support for up to 64K I/O queues, with each I/O queue supporting up to 64K commands.
  • Priority associated with each I/O queue with well-defined arbitration mechanism.
  • All information to complete a 4KB read request is included in the 64B command itself, ensuring efficient small random I/O operation.
  • Efficient and streamlined command set.
  • Support for MSI/MSI-X and interrupt aggregation.
  • Support for multiple namespaces.
  • Efficient support for I/O virtualization architectures like SR-IOV.
  • Robust error reporting and management capabilities.
  • Enterprise: Support for end-to-end data protection (i.e., DIF/DIX).
  • Enterprise: Support for multi-path I/O, including reservations.

NVM Express specifications are owned and maintained by NVM Express, Inc. NVM Express, Inc. was formed as an industry association to define and develop a new storage interface protocol to enable the full performance potential provided by non-volatile memory storage technology, such as PCIe SSDs. NVM Express, Inc. promotes industry awareness of the NVM Express as an industry-wide standard to enable broad adoption and storage device interoperability.