- 0. Overview
- 1. Comment Management
- 2. AppID Management
- 3. VendorID Management
- 4. Combined Management
- 5. Special Metadata File Types
- 6. Metadata-Only Packages
- 6.1 Metadata-Only Package Definition
- 6.2 Valid Use Cases
- 6.3 Security Considerations
- 6.4 Metadata-Only Package API
- 6.4.1 Package IsMetadataOnlyPackage Method
- 6.4.2 Package AddMetadataOnlyFile Method
- 6.4.3 Package GetMetadataOnlyFiles Method
- 6.4.4 Package ValidateMetadataOnlyIntegrity Method
- 6.4.5 Metadata-Only Package Validation
- 6.4.6 Write Operation Requirements
- 6.4.7 Security Considerations for Metadata-Only Packages
- 7. Package Information Structures
- 8. PathMetadata System
- 8.1 PathMetadata Structures
- 8.1.1 PathMetadataType Type
- 8.1.2 PathMetadataEntry Structure
- 8.1.3 PathInheritance Structure
- 8.1.4 PathMetadata Structure
- 8.1.5 PathFileSystem Structure
- 8.1.6 ACLEntry Structure
- 8.1.7 PathMetadataEntry Tag Management
- 8.1.8 PathMetadataEntry Methods
- 8.1.9 PathMetadataEntry Validation Methods
- 8.1.10
PathInfoStructure - 8.1.11
FilePathAssociationStructure - 8.1.12
DestPathOverrideStructure - 8.1.13
DestPathInputInterface
- 8.2
PathMetadataManagement Methods- 8.2.1 Core
PathMetadataCRUD Operations - Path Information Query Methods
- Path Association Methods
- Special Metadata File Management
- Special Metadata File Creation Helpers
- 8.2.2 Package GetPathInfo Method
- 8.2.3 Package ListPaths Method
- 8.2.4 Package ListDirectories Method
- 8.2.5 Package GetDirectoryCount Method
- 8.2.6 Package GetPathHierarchy Method
- 8.2.7 Package AssociateFileWithPath Method
- 8.2.8 Package UpdateFilePathAssociations Method
- 8.2.1 Core
- 8.3 Special Metadata File Management
- 8.4 Path Association System
- 8.5 File-Path Association
- 8.5.1 File-Path Association Query Methods
- 8.5.2 Path Hierarchy Analysis Methods
- 8.5.3 Symbolic Link Management Methods
- 8.5.4 Symlink Validation Methods
- 8.5.5 PathTree Structure
- 8.5.6 PathNode Structure
- 8.5.7 PathStats Structure
- 8.5.8 SymlinkEntry Structure
- 8.5.9 SymlinkMetadata Structure
- 8.5.10 SymlinkFileSystem Structure
- 8.5.11 Symlink Validation Methods (Details)
- 8.5.12 Symlink Creation from Duplicate Paths
- 8.1 PathMetadata Structures
This document defines the package metadata API for the NovusPack system, including comment management, AppID/VendorID operations, special metadata file types, and security validation for metadata fields.
- Go API Definitions Index - Complete index of all Go API functions, types, and structures
- Core Package Interface - Package operations and compression
- Security and Encryption - Comprehensive security architecture and encryption implementation
- File Format Specifications - .nvpk format structure and signature implementation
- Metadata System - Format/schema and special file definitions (Source of Truth)
This section describes comment management operations for packages.
This section describes package-level comment management methods.
// SetComment sets or updates the package comment
// Returns *PackageError on failure
func (p *Package) SetComment(comment string) error// GetComment retrieves the current package comment
func (p *Package) GetComment() string// ClearComment removes the package comment
// Returns *PackageError on failure
func (p *Package) ClearComment() error// HasComment checks if the package has a comment
func (p *Package) HasComment() bool// PackageComment represents the optional package comment section
type PackageComment struct {
CommentLength uint32 // Length of comment including null terminator
Comment string // UTF-8 encoded comment string (null-terminated)
Reserved [3]uint8 // Reserved for future use (must be 0)
}Size: Variable (4 + comment_length + 3 bytes)
Purpose: Provides structured storage for package comment data with validation and serialization support.
This section describes PackageComment methods for comment operations.
// Size returns the size of the package comment
func (pc *PackageComment) Size() int// WriteTo writes the comment to a writer
func (pc *PackageComment) WriteTo(w io.Writer) (int64, error)// ReadFrom reads the comment from a reader
func (pc *PackageComment) ReadFrom(r io.Reader) (int64, error)// Validate validates the package comment
// Returns *PackageError on failure
func (pc *PackageComment) Validate() errorPurpose: Provides low-level access to package comment data and serialization.
Size Returns: int indicating the size of the comment in bytes
w: Writer to write comment data to
WriteTo Returns: Number of bytes written and error
r: Reader to read comment data from
ReadFrom Returns: Number of bytes read and error
Validate Returns: Error if comment is invalid
ErrTypeValidation: Comment format is invalid, comment exceeds size limitsErrTypeIO: I/O error during read/write operations
comment := &PackageComment{...}
// Get comment size
size := comment.Size()
fmt.Printf("Comment size: %d bytes\n", size)
// Write comment to file
file, err := os.Create("comment.txt")
if err != nil {
return err
}
defer file.Close()
bytesWritten, err := comment.WriteTo(file)
if err != nil {
return fmt.Errorf("failed to write comment: %w", err)
}
// Read comment from file
file, err := os.Open("comment.txt")
if err != nil {
return err
}
defer file.Close()
bytesRead, err := comment.ReadFrom(file)
if err != nil {
return fmt.Errorf("failed to read comment: %w", err)
}
// Validate comment
err = comment.Validate()
if err != nil {
return fmt.Errorf("invalid comment: %w", err)
}Creates a new PackageComment with proper initialization.
// NewPackageComment creates and returns a new PackageComment with zero values
func NewPackageComment() *PackageCommentReturns a new PackageComment instance with all fields initialized to their zero values:
CommentLengthset to 0Commentset to empty stringReservedbytes all set to 0
This is equivalent to an empty comment state and is the primary way to create a new PackageComment instance.
Note: For unmarshaling PackageComment instances from binary data, see ReadFrom in the PackageComment Methods section.
This section describes comment security validation functions.
// ValidateComment validates comment content for security issues
// Returns *PackageError on failure
func ValidateComment(comment string) error// SanitizeComment sanitizes comment content to prevent injection attacks
// Returns *PackageError on failure
func SanitizeComment(comment string) (string, error)// ValidateCommentEncoding validates UTF-8 encoding of comment
// Returns *PackageError on failure
func ValidateCommentEncoding(comment []byte) error// CheckCommentLength validates comment length against limits
// Returns *PackageError on failure
func CheckCommentLength(comment string) error// DetectInjectionPatterns scans comment for malicious patterns
func DetectInjectionPatterns(comment string) ([]string, error)This section describes signature comment security features.
// ValidateSignatureComment validates signature comment for security issues
// Returns *PackageError on failure
func ValidateSignatureComment(comment string) error// SanitizeSignatureComment sanitizes signature comment content
// Returns *PackageError on failure
func SanitizeSignatureComment(comment string) (string, error)// CheckSignatureCommentLength validates signature comment length
// Returns *PackageError on failure
func CheckSignatureCommentLength(comment string) error// AuditSignatureComment logs signature comment for security auditing
// Returns *PackageError on failure
func AuditSignatureComment(comment string, signatureIndex int) errorThis section describes AppID management operations.
// SetAppID sets or updates the package AppID
// Returns *PackageError on failure
func (p *Package) SetAppID(appID uint64) error// GetAppID retrieves the current package AppID
func (p *Package) GetAppID() uint64// ClearAppID removes the package AppID (set to 0)
// Returns *PackageError on failure
func (p *Package) ClearAppID() error// HasAppID checks if the package has an AppID (non-zero)
func (p *Package) HasAppID() bool// GetAppIDInfo gets detailed AppID information if available
func (p *Package) GetAppIDInfo() AppIDInfoThis section describes VendorID management operations.
// SetVendorID sets or updates the package VendorID
// Returns *PackageError on failure
func (p *Package) SetVendorID(vendorID uint32) error// GetVendorID retrieves the current package VendorID
func (p *Package) GetVendorID() uint32// ClearVendorID removes the package VendorID (set to 0)
// Returns *PackageError on failure
func (p *Package) ClearVendorID() error// HasVendorID checks if the package has a VendorID (non-zero)
func (p *Package) HasVendorID() bool// GetVendorIDInfo gets detailed VendorID information if available
func (p *Package) GetVendorIDInfo() VendorIDInfoThis section describes combined package identity management operations.
// SetPackageIdentity sets both VendorID and AppID
// Returns *PackageError on failure
func (p *Package) SetPackageIdentity(vendorID uint32, appID uint64) error// GetPackageIdentity gets both VendorID and AppID
func (p *Package) GetPackageIdentity() (uint32, uint64)// ClearPackageIdentity clears both VendorID and AppID
// Returns *PackageError on failure
func (p *Package) ClearPackageIdentity() errorSee Package Information Methods for GetPackageInfo.
NovusPack supports special metadata file types (see File Types System - Special Files) that provide structured metadata and package management capabilities.
This section describes the package metadata file type and operations.
// AddMetadataFile adds a YAML metadata file to the package
// Returns *PackageError on failure
func (p *Package) AddMetadataFile(metadataYAML []byte) error// GetMetadataFile retrieves metadata from the special metadata file
// Returns *PackageError on failure
func (p *Package) GetMetadataFile() ([]byte, error)// UpdateMetadataFile updates the package metadata file
// Returns *PackageError on failure
func (p *Package) UpdateMetadataFile(metadataYAML []byte) error// RemoveMetadataFile removes the package metadata file
// Returns *PackageError on failure
func (p *Package) RemoveMetadataFile() error// HasMetadataFile checks if package has a metadata file
func (p *Package) HasMetadataFile() boolPurpose: Contains structured YAML metadata about the package including:
- Package description and version information
- Author and license details
- Build and compilation metadata
- Custom package-specific data
This section describes the package manifest file type and operations.
// AddManifestFile adds a package manifest file
// Returns *PackageError on failure
func (p *Package) AddManifestFile(manifest ManifestData) error// GetManifestFile retrieves the package manifest
// Returns *PackageError on failure
func (p *Package) GetManifestFile() (ManifestData, error)// UpdateManifestFile updates the package manifest
// Returns *PackageError on failure
func (p *Package) UpdateManifestFile(updates ManifestData) error// RemoveManifestFile removes the package manifest
// Returns *PackageError on failure
func (p *Package) RemoveManifestFile() error// HasManifestFile checks if package has a manifest file
func (p *Package) HasManifestFile() boolPurpose: Defines the package structure and dependencies including:
- File organization and structure
- Dependency requirements
- Installation instructions
- Package relationships
This section describes the package index file type and operations.
// AddIndexFile adds a package index file
// Returns *PackageError on failure
func (p *Package) AddIndexFile(index IndexData) error// GetIndexFile retrieves the package index
// Returns *PackageError on failure
func (p *Package) GetIndexFile() (IndexData, error)// UpdateIndexFile updates the package index
// Returns *PackageError on failure
func (p *Package) UpdateIndexFile(updates IndexData) error// RemoveIndexFile removes the package index
// Returns *PackageError on failure
func (p *Package) RemoveIndexFile() error// HasIndexFile checks if package has an index file
func (p *Package) HasIndexFile() boolPurpose: Provides file navigation and indexing including:
- File location mappings
- Content-based indexing
- Search and navigation data
- File relationship mappings
This section describes the package signature file type and operations.
// AddSignatureFile adds a digital signature file
// Returns *PackageError on failure
func (p *Package) AddSignatureFile(signature SignatureData) error// GetSignatureFile retrieves the signature file
// Returns *PackageError on failure
func (p *Package) GetSignatureFile() (SignatureData, error)// UpdateSignatureFile updates the signature file
// Returns *PackageError on failure
func (p *Package) UpdateSignatureFile(updates SignatureData) error// RemoveSignatureFile removes the signature file
// Returns *PackageError on failure
func (p *Package) RemoveSignatureFile() error// HasSignatureFile checks if package has a signature file
func (p *Package) HasSignatureFile() boolPurpose: Contains digital signature information including:
- Signature metadata and timestamps
- Public key information
- Signature validation data
- Trust chain information
This section describes special file management operations.
// GetSpecialFiles returns all special files in the package
func (p *Package) GetSpecialFiles() ([]SpecialFileInfo, error)// GetSpecialFileByType retrieves special file by type
func (p *Package) GetSpecialFileByType(fileType FileType) (SpecialFileInfo, error)// RemoveSpecialFile removes a special file by type
// Returns *PackageError on failure
func (p *Package) RemoveSpecialFile(fileType FileType) error// ValidateSpecialFiles validates all special files
// Returns *PackageError on failure
func (p *Package) ValidateSpecialFiles() errorThis section describes special file data structures used in the API.
// SpecialFileInfo contains information about special metadata files in the package.
type SpecialFileInfo struct {
Type FileType // File type (see [File Types System - Special Files](file_type_system.md#339-special-file-types-65000-65535))
Name string // Special file name (e.g., "__NVPK_META_240__.nvpkmeta")
Size int64 // File size in bytes
Offset int64 // Offset in package
Data []byte // File content
Valid bool // Whether file is valid
Error string // Error message if invalid
}// ManifestData contains manifest file data structure.
type ManifestData struct {
Version string // Manifest version
Package PackageInfo // Package information
Dependencies []Dependency // Package dependencies
Structure []FileStructure // File organization
Install InstallInfo // Installation instructions
}// IndexData contains index file data structure.
type IndexData struct {
Version string // Index version
Files []FileIndex // File index entries
Navigation NavigationData // Navigation structure
Search SearchIndex // Search index data
}// SignatureData contains signature file data structure.
type SignatureData struct {
Version string // Signature format version
Signatures []SignatureInfo // Signature information
TrustChain []TrustInfo // Trust chain data
Validation ValidationData // Validation metadata
}Metadata-only packages are NovusPack packages that contain no regular content files (FileCount = 0). These packages may contain special metadata files (see File Types System - Special Files) or may be completely empty. They serve specific purposes in package management and distribution systems, including placeholders and namespace reservations.
A metadata-only package is defined as:
- FileCount = 0: No regular content files
- IsMetadataOnly flag = 1: Package header Bit 7 (metadata-only) is set
- TotalSize = 0: No uncompressed content data
Metadata-only packages may or may not contain special metadata files. Packages with 0 files but no special metadata files are valid and represent empty or placeholder packages.
This section describes valid use cases for metadata-only packages.
- Package listings: Catalogs of available packages with metadata
- Dependency resolution: Packages defining dependency trees
- Package discovery: Searchable indexes of package repositories
- Configuration templates: Packages containing configuration schemas
- API specifications: Packages with API definitions and schemas
- Data structure definitions: Packages containing only data models
- Update manifests: Packages describing updates without actual files
- Installation scripts: Packages containing installation instructions
- Package relationships: Packages defining inter-package relationships
- Build configurations: Packages containing build system configurations
- Development metadata: Packages with development environment specifications
- Testing configurations: Packages containing test specifications
- Empty packages: Packages with no files or metadata, used as placeholders
- Reserved packages: Packages claiming a namespace or identifier
- Future expansion: Packages created for future content addition
This section describes security considerations for metadata-only packages.
- Metadata integrity: Signatures must validate all metadata files
- Empty content handling: Special validation for packages with no content
- Signature scope: Clear definition of what gets signed in metadata-only packages
- Content verification: No actual content to verify, trust relies on metadata
- Metadata tampering: Risk of metadata manipulation without content cross-reference
- Trust chain: Enhanced trust requirements for metadata-only packages
- Size validation: Very small packages require enhanced validation
- Structure validation: Ensure package structure is valid without content
- Metadata consistency: Verify metadata files are internally consistent
- Metadata injection: Potential for malicious metadata injection
- Dependency confusion: Risk of redirecting dependencies maliciously
- Trust abuse: Exploiting trust in metadata-only packages
This section describes the metadata-only package API.
// IsMetadataOnlyPackage checks if package contains only metadata files
func (p *Package) IsMetadataOnlyPackage() bool// AddMetadataOnlyFile adds a special metadata file to a metadata-only package
// Returns *PackageError on failure
func (p *Package) AddMetadataOnlyFile(fileType FileType, data []byte) error// GetMetadataOnlyFiles returns all metadata files in the package
// Returns *PackageError on failure
func (p *Package) GetMetadataOnlyFiles() ([]SpecialFileInfo, error)// ValidateMetadataOnlyIntegrity validates metadata-only package integrity
// Returns *PackageError on failure
func (p *Package) ValidateMetadataOnlyIntegrity() errorSee 6.4.5 Package.ValidateMetadataOnlyPackage Method for ValidateMetadataOnlyPackage.
// ValidateMetadataOnlyPackage performs comprehensive validation:
// Ensure FileCount == 0
// Ensure IsMetadataOnly flag (Bit 7) is set in header
// Validate all special metadata files (if present)
// Check for malicious metadata patterns (if metadata files present)
// Verify signature scope includes all metadata (if signatures present)
// Ensure metadata consistency
// Validate package structure
func (p *Package) ValidateMetadataOnlyPackage() errorWrite operations automatically determine and set the metadata-only flag based on package state:
- Automatic Detection: Write operations automatically detect metadata-only status when FileCount = 0 (no regular content files)
- Header Flag: Write operations MUST set the metadata-only flag (Bit 7) when FileCount = 0
- Flag Synchronization: PackageInfo.IsMetadataOnly MUST be synchronized to header flag before writing
- Validation: Write operations MUST validate that FileCount = 0 matches IsMetadataOnly flag state
- Empty Packages: Empty packages with no special metadata files are valid and automatically marked as metadata-only
Note: There is no separate function to create metadata-only packages. Metadata-only status is determined automatically during write operations based on FileCount. To create a metadata-only package, create a package with no regular files using standard package creation methods.
- Optional signatures: Signatures are recommended but not mandatory for metadata-only packages
- Enhanced validation: Stricter validation requirements for packages with special metadata files
- Trust verification: Higher trust requirements for metadata-only packages with special metadata
- Audit logging: Enhanced logging for metadata-only package operations with special metadata
This section describes package information structures used in the API.
The PackageInfo structure provides comprehensive package information and metadata:
// PackageInfo contains comprehensive package information and metadata.
type PackageInfo struct {
// Basic Package Information
FormatVersion uint32 // Package file format version
FileCount int // Number of regular content files in the package (excludes special metadata files, types 65000-65535)
FilesUncompressedSize int64 // Total uncompressed size of all regular files (excludes special metadata files)
FilesCompressedSize int64 // Total compressed size of all regular files (excludes special metadata files)
// Package Identity
VendorID uint32 // Vendor/platform identifier
AppID uint64 // Application identifier
// Package Comment
HasComment bool // Whether package has a comment
Comment string // Actual package comment content
// Digital Signatures (Multiple Signatures Support)
HasSignatures bool // Whether package has any signatures
SignatureCount int // Number of signatures in the package
Signatures []SignatureInfo // Detailed signature information
// Security Information
IsImmutable bool // Whether package is immutable (signed)
// Timestamps
Created time.Time // Package creation timestamp
Modified time.Time // Package modification timestamp
// Version Tracking
PackageDataVersion uint32 // Tracks changes to package data content (file additions, removals, data modifications)
MetadataVersion uint32 // Tracks changes to package metadata (comment and identity changes)
// Package Features
HasMetadataFiles bool // Whether package has metadata files
HasPerFileTags bool // Whether package has per-file tags (path metadata with properties)
HasExtendedAttrs bool // Whether package has extended attributes (filesystem metadata)
HasEncryptedData bool // Whether package contains encrypted files
HasCompressedData bool // Whether package contains compressed files
IsMetadataOnly bool // Whether package contains no regular files (FileCount = 0)
// Package Compression
PackageCompression uint8 // Package compression type (0=none, 1=Zstd, 2=LZ4, 3=LZMA)
IsPackageCompressed bool // Whether the entire package is compressed
PackageOriginalSize int64 // Original package size before compression (0 if not compressed)
PackageCompressedSize int64 // Compressed package size (0 if not compressed)
PackageCompressionRatio float64 // Compression ratio (0.0-1.0, 0.0 if not compressed)
}PackageInfo provides lightweight package-level information and does NOT include:
- Individual
FileEntrymetadata (useListFiles()orGetMetadata()for file-level details) - Special metadata file contents (use
GetMetadata()for special file data) - Path metadata entries (use
GetMetadata()for path metadata) - Full signature data (only signature summary information is included; see
Signaturesfield forSignatureInfodetails)
For comprehensive metadata including file entries and special metadata files, use GetMetadata() which returns PackageMetadata containing PackageInfo plus all file and metadata details.
Creates a new PackageInfo with proper initialization.
// NewPackageInfo creates a new PackageInfo with default values
func NewPackageInfo() *PackageInfoReturns a new PackageInfo instance with all fields initialized to their zero values or specification defaults:
- All numeric fields set to 0
- All boolean fields set to false
- String fields set to empty strings
- Slice fields initialized to empty slices
- Time fields set to zero time (will be set on package creation)
There is no package-wide security level.
This ensures that all PackageInfo initialization is centralized and aligned with specification defaults.
Note: PackageInfo instances are typically created by the package system during package creation or opening. This factory function provides a consistent way to initialize PackageInfo instances when needed.
PackageInfo serves different roles depending on the operation phase:
-
During Package Open/Read Operations:
- Package header is read from disk (header is the on-disk source of truth)
- Header flags and metadata are used to populate PackageInfo fields
- PackageInfo is synchronized from header state
-
During In-Memory Operations:
- PackageInfo is the source of truth for all package metadata and flags
- All package operations read from and update PackageInfo
- Header fields may become stale during in-memory operations
-
During Package Write/Save Operations:
- The header is synchronized from PackageInfo before serialization.
- PackageInfo values are written to header fields.
- PackageInfo versions are written to the header version fields.
Flag Synchronization: Functions like UpdateSpecialMetadataFlags() currently update both header flags and PackageInfo fields simultaneously during in-memory operations. This ensures consistency, but the architectural intent is that PackageInfo should be the source of truth during in-memory operations, with the header being updated from PackageInfo during write operations.
Synchronizes a PackageInfo instance from an on-disk PackageHeader.
This helper provides the standard, centralized mapping from header fields and flags to PackageInfo fields.
// FromHeader synchronizes PackageInfo fields from the provided PackageHeader.
//
// This method must only copy data that is represented in the header.
// It must not compute derived values that require scanning file entries or reading file data.
//
// Returns *PackageError on failure.
func (pi *PackageInfo) FromHeader(header *PackageHeader) errorFromHeader must synchronize at least the following fields:
- FormatVersion from header.FormatVersion.
- VendorID from header.VendorID.
- AppID from header.AppID.
- PackageDataVersion from header.PackageDataVersion.
- MetadataVersion from header.MetadataVersion.
- Created from header.CreatedTime.
- Modified from header.ModifiedTime.
- HasComment from header.CommentSize (or equivalent comment presence indicator).
- HasSignatures and IsImmutable from header.SignatureOffset (signed packages are immutable).
- PackageCompression and IsPackageCompressed from the compression type encoded in header.Flags (bits 15-8).
- Feature flags derived from header.Flags (bits 0-7), including HasMetadataFiles, HasPerFileTags, HasExtendedAttrs, HasEncryptedData, HasCompressedData, and IsMetadataOnly.
// PackageHeader represents the fixed-size header of a NovusPack (.nvpk) file
// Size: 112 bytes (fixed)
type PackageHeader struct {
Magic uint32 // Package identifier (0x4E56504B "NVPK")
FormatVersion uint32 // Format version (current: 1)
Flags uint32 // Package-level features and options
PackageDataVersion uint32 // Tracks changes to package data content
MetadataVersion uint32 // Tracks changes to package metadata
PackageCRC uint32 // CRC32 of package content (0 if skipped)
CreatedTime uint64 // Package creation timestamp (Unix nanoseconds)
ModifiedTime uint64 // Package modification timestamp (Unix nanoseconds)
LocaleID uint32 // Locale identifier for path encoding
Reserved uint32 // Reserved for future use (must be 0)
AppID uint64 // Application/game identifier (0 if not associated)
VendorID uint32 // Storefront/platform identifier (0 if not associated)
CreatorID uint32 // Creator identifier (reserved for future use)
IndexStart uint64 // Offset to file index from start of file
IndexSize uint64 // Size of file index in bytes
ArchiveChainID uint64 // Archive chain identifier
ArchivePartInfo uint32 // Combined part number and total parts
CommentSize uint32 // Size of package comment in bytes (0 if no comment)
CommentStart uint64 // Offset to package comment from start of file
SignatureOffset uint64 // Offset to signatures block from start of file
}Purpose: Provides comprehensive metadata and navigation information for the entire package.
Cross-Reference: See Package File Format - Package Header for complete header specification.
Synchronizes an on-disk PackageHeader instance from an in-memory PackageInfo instance.
This helper provides the standard, centralized mapping from PackageInfo fields to the header fields and flags written to disk.
// ToHeader synchronizes PackageHeader fields from the provided PackageInfo.
//
// This method must only write fields that are represented in the header.
// It must not mutate fields that are computed by the writer pipeline (for example IndexStart, IndexSize, and CRC).
//
// Returns *PackageError on failure.
func (h *PackageHeader) ToHeader(pi *PackageInfo) errorToHeader must synchronize at least the following fields:
- FormatVersion from pi.FormatVersion.
- VendorID from pi.VendorID.
- AppID from pi.AppID.
- PackageDataVersion from pi.PackageDataVersion.
- MetadataVersion from pi.MetadataVersion.
- CreatedTime from pi.Created.
- ModifiedTime from pi.Modified.
- Compression type bits (15-8) from pi.PackageCompression.
- Feature flags (bits 0-7) from the corresponding PackageInfo booleans.
- CommentSize and CommentStart must reflect the serialized comment section computed by the writer.
- SignatureOffset must reflect the serialized signatures section computed by the signer.
// SignatureInfo contains signature information for a package.
type SignatureInfo struct {
Index int // Signature index in the package
Type uint32 // Signature type (ML-DSA, SLH-DSA, PGP, X.509)
Size uint32 // Size of signature data in bytes
Offset uint64 // Offset to signature data from start of file
Flags uint32 // Signature-specific flags
Timestamp uint32 // Unix timestamp when signature was created
Comment string // Signature comment (if any)
Algorithm string // Algorithm name/description
SecurityLevel int // Algorithm security level (v2, signature algorithm specific)
Valid bool // Whether signature is valid
Trusted bool // Whether signature is trusted
Error string // Error message if validation failed
}// SecurityStatus contains the security status of a package.
type SecurityStatus struct {
SignatureCount int // Number of signatures
ValidSignatures int // Number of valid signatures
TrustedSignatures int // Number of trusted signatures
SignatureResults []SignatureValidationResult // Individual results
HasChecksums bool // Checksums present
ChecksumsValid bool // Checksums valid
ValidationErrors []string // Validation errors
}This section describes package information methods.
// GetPackageInfo returns comprehensive package information
func (p *Package) GetPackageInfo() (*PackageInfo, error)// RefreshPackageInfo refreshes package information from the file on-disk
// Returns *PackageError on failure
func (p *Package) RefreshPackageInfo(ctx context.Context) errorSee GetSecurityStatus for the security status method.
The path metadata system provides structured path definitions (files, directories, and symlinks) with metadata, inheritance rules, and filesystem properties for the NovusPack system using special metadata files.
Tag Inheritance Model: Tag inheritance only works with PathMetadataEntry instances, not FileEntry instances directly.
This is because FileEntry can have multiple paths (via FileEntry.Paths), while PathMetadataEntry represents a single path.
Inheritance is resolved per-path by walking up the PathMetadataEntry.ParentPath chain.
Each PathMetadataEntry can have its own inheritance chain, allowing different paths for the same file content to inherit different tags.
Cross-Reference: For file-path association methods and FileEntry path operations, see FileEntry API - Path Management.
Note: These methods assume the following imports:
import (
"context"
"encoding/json"
"fmt"
"strconv"
"strings"
"time"
"gopkg.in/yaml.v3"
"github.com/google/uuid"
)
// Tag, TagValueType from api_file_managementNote: PathEntry is defined in the generics package and is shared by both FileEntry and PathMetadataEntry. See Generic Types and Patterns - PathEntry for complete specification.
// PathMetadataType represents the type of path entry
type PathMetadataType uint8
const (
PathMetadataTypeFile PathMetadataType = 0 // Regular file
PathMetadataTypeDirectory PathMetadataType = 1 // Regular directory
PathMetadataTypeFileSymlink PathMetadataType = 2 // Symlink to a file
PathMetadataTypeDirectorySymlink PathMetadataType = 3 // Symlink to a directory
)// PathMetadataEntry represents a path (file, directory, or symlink) with metadata, inheritance rules, and filesystem properties
type PathMetadataEntry struct {
// Path is the path entry (minimal PathEntry from generics package)
// All paths are stored with a leading "/" to indicate the package root
// - Directory paths must end with "/" (e.g., "/assets/")
// - File paths must NOT end with "/" (e.g., "/assets/file.txt")
// - Root path is represented as "/" (the package root)
// - All paths use forward slashes ("/") as separators
Path generics.PathEntry `yaml:"path"`
// Type indicates whether this is a file, directory, file symlink, or directory symlink
Type PathMetadataType `yaml:"type"`
// Tags are path-specific tags (typed tags)
Tags []*generics.Tag[any] `yaml:"tags"`
// Inheritance controls tag inheritance behavior (optional, only for directories)
Inheritance *PathInheritance `yaml:"inheritance,omitempty"` // Inheritance settings (nil for files)
// Metadata contains path metadata (optional, only for directories)
Metadata *PathMetadata `yaml:"metadata,omitempty"` // Path metadata (nil for files)
// Destination extraction override paths.
//
// These fields define optional destination extraction directory overrides for this path.
// They are interpreted by ExtractPath destination resolution.
//
// Destinations may be absolute or relative.
// Relative destinations may include "." and ".." segments.
//
// Relative destinations are resolved relative to the default extraction directory for the path.
// This is the directory the path would be extracted to under session base with no overrides.
//
// Cross-Reference: See [File Extraction API - ExtractPath Destination Resolution](api_file_mgmt_extraction.md#151-extractpath-destination-resolution).
DestPath string `yaml:"dest_path,omitempty"`
DestPathWin string `yaml:"dest_path_win,omitempty"`
// FileSystem contains filesystem-specific properties
FileSystem PathFileSystem `yaml:"filesystem"`
// Path hierarchy (runtime only, not stored in file)
ParentPath *PathMetadataEntry `yaml:"-"` // Pointer to parent path (nil for root)
// FileEntry associations (runtime only, not stored in file)
// Since FileEntry and PathMetadataEntry are in the same package, direct references are used.
AssociatedFileEntries []*FileEntry `yaml:"-"` // FileEntry instances associated with this path
}// PathInheritance controls tag inheritance behavior (for directories only)
type PathInheritance struct {
Enabled bool `yaml:"enabled"` // Whether this path provides inheritance
Priority int `yaml:"priority"` // Inheritance priority (higher = more specific)
}// PathMetadata contains path metadata (for directories only)
type PathMetadata struct {
Created string `yaml:"created"` // Path creation time (ISO8601)
Modified string `yaml:"modified"` // Last modification time (ISO8601)
Description string `yaml:"description"` // Human-readable description
}// PathFileSystem contains filesystem-specific properties
type PathFileSystem struct {
// Execute permissions (always captured)
IsExecutable bool `yaml:"is_executable"` // Whether file has any execute permission bits set (tracked by default)
// Unix/Linux properties (optional, captured when PreservePermissions is enabled)
Mode *uint32 `yaml:"mode,omitempty"` // File/directory permissions and type (Unix-style)
UID *uint32 `yaml:"uid,omitempty"` // User ID
GID *uint32 `yaml:"gid,omitempty"` // Group ID
ACL []ACLEntry `yaml:"acl,omitempty"` // Access Control List
// Timestamps (Unix nanoseconds since epoch)
ModTime uint64 `yaml:"mod_time,omitempty"` // Modification time
CreateTime uint64 `yaml:"create_time,omitempty"` // Creation time
AccessTime uint64 `yaml:"access_time,omitempty"` // Access time
// Symbolic link support
LinkTarget string `yaml:"link_target,omitempty"` // Target path for symbolic links (empty if not a symlink)
// Windows properties (optional)
WindowsAttrs *uint32 `yaml:"windows_attrs,omitempty"` // Windows attributes
// Extended attributes (optional)
ExtendedAttrs map[string]string `yaml:"extended_attrs,omitempty"` // Extended attributes
// Filesystem flags (optional)
Flags *uint16 `yaml:"flags,omitempty"` // Filesystem-specific flags
}// ACLEntry represents an Access Control List entry
type ACLEntry struct {
Type string `yaml:"type"` // "user", "group", "other", "mask"
ID *uint32 `yaml:"id,omitempty"` // User/Group ID (nil for "other")
Perms string `yaml:"perms"` // Permissions (e.g., "rwx", "r--")
}All tag operations use typed tags for type safety. See FileEntry API - Tag Management for complete tag system documentation.
Tag Management Architecture: Tags are managed directly on PathMetadataEntry and FileEntry instances.
There are no Package-level tag management functions.
When deciding where to set tags, consider the following guidance:
- Set tags on
FileEntrywhen the same tags need to apply to all paths for a file with multiple paths. - Set tags on
PathMetadataEntrywhen tags should only apply to a specific path. PathMetadataEntrytags can participate in inheritance;FileEntrytags do not.
Note: These are standalone functions rather than methods due to Go's limitation of not supporting generic methods on non-generic types. See Generic Types and Patterns for details.
// GetPathMetaTags returns all tags as typed tags for a PathMetadataEntry
// Returns *PackageError on failure
func GetPathMetaTags(pme *PathMetadataEntry) ([]*Tag[any], error)// GetPathMetaTagsByType returns all tags of a specific type for a PathMetadataEntry
// Returns a slice of Tag pointers with the specified type parameter T
// Only tags matching the type T and corresponding TagValueType are returned
// Returns *PackageError on failure
func GetPathMetaTagsByType[T any](pme *PathMetadataEntry) ([]*Tag[T], error)// AddPathMetaTags adds multiple new tags with type safety to a PathMetadataEntry
// Returns *PackageError if any tag with the same key already exists
func AddPathMetaTags(pme *PathMetadataEntry, tags []*Tag[any]) error// SetPathMetaTags updates existing tags from a slice of typed tags for a PathMetadataEntry
// Returns *PackageError if any tag key does not already exist
// Only modifies tags that already exist; does not create new tags
func SetPathMetaTags(pme *PathMetadataEntry, tags []*Tag[any]) error// GetPathMetaTag retrieves a type-safe tag by key from a PathMetadataEntry
// Returns the tag pointer and an error. If the tag is not found, returns (nil, nil).
// If an underlying error occurs, returns (nil, error).
// Returns *PackageError on failure
// If the tag type is unknown, use GetPathMetaTag[any](pme, "key") to retrieve the tag and inspect its Type field
func GetPathMetaTag[T any](pme *PathMetadataEntry, key string) (*Tag[T], error)// AddPathMetaTag adds a new tag with type safety to a PathMetadataEntry
// Returns *PackageError if a tag with the same key already exists
func AddPathMetaTag[T any](pme *PathMetadataEntry, key string, value T, tagType TagValueType) error// SetPathMetaTag updates an existing tag with type safety for a PathMetadataEntry
// Returns *PackageError if the tag key does not already exist
// Only modifies existing tags; does not create new tags
func SetPathMetaTag[T any](pme *PathMetadataEntry, key string, value T, tagType TagValueType) error// RemovePathMetaTag removes a tag by key from a PathMetadataEntry
// Returns *PackageError on failure
func RemovePathMetaTag(pme *PathMetadataEntry, key string) error// HasPathMetaTag checks if a tag with the specified key exists on a PathMetadataEntry
func HasPathMetaTag(pme *PathMetadataEntry, key string) boolThis section describes PathMetadataEntry methods.
// Path management methods for PathMetadataEntry
func (pme *PathMetadataEntry) SetPath(path string)// GetPath returns the path as stored (Unix-style with forward slashes).
// For platform-specific display, use GetPathForPlatform() or convert manually.
func (pme *PathMetadataEntry) GetPath() string// GetPathForPlatform returns the path converted for the specified platform.
// On Windows: converts forward slashes to backslashes.
// On Unix/Linux: returns the path as stored (with forward slashes).
func (pme *PathMetadataEntry) GetPathForPlatform(isWindows bool) string// GetPathEntry returns the PathEntry representation of this path metadata entry.
func (pme *PathMetadataEntry) GetPathEntry() generics.PathEntry// Type and symlink methods for PathMetadataEntry
func (pme *PathMetadataEntry) GetType() PathMetadataType// IsDirectory returns true if this path metadata entry represents a directory.
func (pme *PathMetadataEntry) IsDirectory() bool// IsFile returns true if this path metadata entry represents a file.
func (pme *PathMetadataEntry) IsFile() bool// IsSymlink returns true if this path metadata entry represents a symlink.
func (pme *PathMetadataEntry) IsSymlink() bool// GetLinkTarget returns the target path of the symlink.
func (pme *PathMetadataEntry) GetLinkTarget() string// ResolveSymlink resolves the symlink to its final target path.
func (pme *PathMetadataEntry) ResolveSymlink() string// Parent path management methods for PathMetadataEntry
func (pme *PathMetadataEntry) SetParentPath(parent *PathMetadataEntry)// GetParentPath returns the parent path metadata entry.
func (pme *PathMetadataEntry) GetParentPath() *PathMetadataEntry// GetParentPathString returns the parent path as a string.
func (pme *PathMetadataEntry) GetParentPathString() string// GetDepth returns the depth of this path in the directory hierarchy.
func (pme *PathMetadataEntry) GetDepth() int// IsRoot returns true if this path metadata entry represents the root path.
func (pme *PathMetadataEntry) IsRoot() bool// GetAncestors returns all ancestor path metadata entries up to the root.
func (pme *PathMetadataEntry) GetAncestors() []*PathMetadataEntry// Tag inheritance methods for PathMetadataEntry
// These methods resolve inheritance by walking up the ParentPath chain
func (pme *PathMetadataEntry) GetInheritedTags() ([]*Tag[any], error)// GetEffectiveTags returns all tags for this PathMetadataEntry, including:
// 1. Tags directly on this PathMetadataEntry
// 2. Tags inherited from parent PathMetadataEntry instances (path hierarchy)
// 3. Tags from associated FileEntry instances (treated as if applied to this PathMetadataEntry)
func (pme *PathMetadataEntry) GetEffectiveTags() ([]*Tag[any], error)// FileEntry association methods for PathMetadataEntry
// AssociateWithFileEntry associates this PathMetadataEntry with a FileEntry
// The association is established if the PathMetadataEntry.Path.Path matches one of the FileEntry.Paths
// Returns *PackageError on failure
func (pme *PathMetadataEntry) AssociateWithFileEntry(fe *FileEntry) error// GetAssociatedFileEntries returns all FileEntry instances associated with this PathMetadataEntry
// Returns empty slice if no FileEntry instances are associated
func (pme *PathMetadataEntry) GetAssociatedFileEntries() []*FileEntryThis section defines validation behavior for PathMetadataEntry instances.
// Validate validates the PathMetadataEntry state and returns an error on failure.
func (pme *PathMetadataEntry) Validate() error// PathInfo provides runtime path metadata information
type PathInfo struct {
Entry PathMetadataEntry // Path metadata entry data
FileCount int // Number of files in this path
SubDirs []string // Immediate subdirectories
ParentPath string // Parent path
Depth int // Path depth (0 = root)
}// FilePathAssociation links files to their path metadata
type FilePathAssociation struct {
FilePath string // File path
Path string // Parent path
PathMetadata *PathInfo // Path metadata information (nil if no path metadata)
InheritedTags []Tag // Tags inherited from path hierarchy
EffectiveTags []Tag // All tags including inheritance
}// DestPathOverride specifies destination extraction directory overrides.
//
// A nil field means "no override specified" for that field.
type DestPathOverride struct {
DestPath *string
DestPathWin *string
}// DestPathInput is the allowed input type set for SetDestPath.
//
// DestPathInput supports:
// - string: a single destination string
// - map[string]string: a map with keys "DestPath" and/or "DestPathWin"
//
// Note: The map form uses string keys for ergonomics in callers.
// Keys other than "DestPath" and "DestPathWin" MUST be rejected with ErrTypeValidation.
type DestPathInput interface {
~string | ~map[string]string
}The PathMetadata management API provides methods for creating, reading, updating, and deleting path metadata entries.
This section describes core CRUD operations for PathMetadata.
// GetPathMetadata retrieves all path metadata entries from the package
// Returns *PackageError on failure
func (p *Package) GetPathMetadata(ctx context.Context) ([]*PathMetadataEntry, error)Returns all path metadata entries currently stored in the package.
// SetPathMetadata replaces all path metadata entries in the package
// Returns *PackageError on failure
func (p *Package) SetPathMetadata(ctx context.Context, entries []*PathMetadataEntry) errorReplaces all existing path metadata entries with the provided entries.
// AddPathMetadata adds a new path metadata entry to the package
// Returns *PackageError on failure
func (p *Package) AddPathMetadata(ctx context.Context, path string, pathType PathMetadataType, properties map[string]string, inheritance *PathInheritance, metadata *PathMetadata) errorAdds a new path metadata entry with the specified path, type, properties, inheritance rules, and metadata.
// RemovePathMetadata removes a path metadata entry by path
// Returns *PackageError on failure
func (p *Package) RemovePathMetadata(ctx context.Context, path string) errorRemoves the path metadata entry for the specified path.
// UpdatePathMetadata updates an existing path metadata entry
// Returns *PackageError on failure
func (p *Package) UpdatePathMetadata(ctx context.Context, path string, properties map[string]string, inheritance *PathInheritance, metadata *PathMetadata) errorUpdates the path metadata entry for the specified path with new properties, inheritance rules, and metadata.
// SetDestPath sets destination extraction directory overrides for a stored path.
//
// This is a pure in-memory operation.
//
// storedPath MUST be treated as a stored package path.
// If storedPath does not begin with "/", the implementation MUST prefix "/" before matching or creating entries.
//
// If no PathMetadataEntry exists for storedPath, SetDestPath MUST create one.
// The new entry type MUST be inferred from storedPath:
// - "/" is PathMetadataTypeDirectory
// - paths ending with "/" are PathMetadataTypeDirectory
// - all other paths are PathMetadataTypeFile
//
// Returns *PackageError on failure.
func (p *Package) SetDestPath(storedPath string, override DestPathOverride) errorSets destination extraction directory overrides for a stored path. This is a pure in-memory operation that creates a PathMetadataEntry if one doesn't exist.
// SetDestPathTyped is a generic helper for SetDestPath.
//
// This helper exists to allow compile-time type checking of the dest input.
// It converts dest to DestPathOverride, then delegates to Package.SetDestPath.
//
// If dest is a string, it MUST be parsed to determine which destination field to set.
// If the string is a Windows-only absolute path (drive letter like "C:\\" or "C:/", or UNC path like "\\\\server\\share"),
// it MUST be stored as DestPathWin.
// Otherwise, it MUST be stored as DestPath.
func SetDestPath[T DestPathInput](p *Package, storedPath string, dest T) errorGeneric helper for SetDestPath that provides compile-time type checking.
These methods are wrappers around AddPathMetadata, RemovePathMetadata, and UpdatePathMetadata.
They specifically handle directory paths (paths ending with /).
// AddDirectoryMetadata adds directory path metadata (metadata-only, does not add files)
// Returns *PackageError on failure
func (p *Package) AddDirectoryMetadata(ctx context.Context, path string, properties map[string]string, inheritance *PathInheritance, metadata *PathMetadata) error// RemoveDirectoryMetadata removes directory path metadata (metadata-only, does not remove files)
// Returns *PackageError on failure
func (p *Package) RemoveDirectoryMetadata(ctx context.Context, path string) error// UpdateDirectoryMetadata updates directory path metadata (metadata-only, does not modify files)
// Returns *PackageError on failure
func (p *Package) UpdateDirectoryMetadata(ctx context.Context, path string, properties map[string]string, inheritance *PathInheritance, metadata *PathMetadata) errorNote: These methods are metadata-only operations. To add or remove files within a directory, use AddDirectory or RemoveDirectory from the File Management API.
// ValidatePathMetadata validates all path metadata entries
// Returns *PackageError on failure
func (p *Package) ValidatePathMetadata() error// GetPathConflicts returns a list of paths with conflicting metadata
// Returns *PackageError on failure
func (p *Package) GetPathConflicts() ([]string, error)Validates path metadata consistency and identifies conflicts.
- GetPathInfo - Returns complete path information for a specific path
- ListPaths - Returns all paths in the package
- ListDirectories - Returns all directory paths
- GetDirectoryCount - Returns the total number of directories
- GetPathHierarchy - Returns the path hierarchy as a map
- AssociateFileWithPath - Links a file to its path metadata
- UpdateFilePathAssociations - Rebuilds all file-path associations
See Path Association System for additional association methods including DisassociateFileFromPath and GetFilePathAssociations.
- SavePathMetadataFile - Creates and saves the path metadata file
- LoadPathMetadataFile - Loads and parses the path metadata file
- UpdateSpecialMetadataFlags - Updates package header flags
See Special Metadata File Management for details.
// CreateSpecialMetadataFile creates a special metadata FileEntry
// Returns *PackageError on failure
func (p *Package) CreateSpecialMetadataFile(ctx context.Context, fileType uint16, fileName string, content []byte) (*FileEntry, error)// UpdateSpecialMetadataFile updates an existing special metadata file
// Returns *PackageError on failure
func (p *Package) UpdateSpecialMetadataFile(ctx context.Context, fileType uint16, fileName string, content []byte) error// RemoveSpecialMetadataFile removes a special metadata file
// Returns *PackageError on failure
func (p *Package) RemoveSpecialMetadataFile(ctx context.Context, fileType uint16, fileName string) errorHelper methods for creating, updating, and removing special metadata files.
Returns complete path information including metadata, inheritance, and associations for a specific path.
path: Path string to query
*PathInfo: Complete path information structureerror: Any error that occurred during the query
Returns all paths in the package with their complete metadata.
[]PathInfo: Slice of all path information entrieserror: Any error that occurred during the query
Returns all directory paths in the package with their complete metadata.
Only returns paths that are marked as directories (paths ending with / or with directory metadata).
[]PathInfo: Slice of directory path information entrieserror: Any error that occurred during the query
- List all directories in the package
- Query directory permissions and metadata
- Build directory tree visualizations
- Verify directory metadata consistency
Returns the total number of directories in the package.
int: Total count of directory pathserror: Any error that occurred during the query
- Package statistics
- Progress tracking during extraction
- Directory structure validation
- Quick checks without loading full path data
Returns the path hierarchy as a map of parent paths to their child paths.
map[string][]string: Map of parent path to slice of child pathserror: Any error that occurred during the query
- Build directory tree structures
- Resolve path inheritance hierarchies
- Validate path relationships
- Efficient traversal of path hierarchies
AssociateFileWithPath links a file to its path metadata, establishing the association between a FileEntry and its corresponding PathMetadataEntry.
The function performs the following operations:
- Validates the context for cancellation/timeout.
- Retrieves the FileEntry by path using
findFileEntryByPath(). - Retrieves the path metadata entry by path using
findPathMetadataByPath(). - Establishes the association by calling
FileEntry.AssociateWithPathMetadata()with the retrieved PathMetadataEntry. - Sets parent path association for hierarchy traversal:
- Derives the parent path using
filepath.Diron the path (after normalizing trailing slashes) - Validates that the parent path is different from the path and not the current directory (
.) - If valid, retrieves the parent path metadata entry and sets
PathMetadataEntry.ParentPathto establish the hierarchy - This operation never fails - missing parents are valid for root paths or when path metadata is optional
ParentPathis a runtime-only convenience field (not persisted) that isnilfor root paths
- Derives the parent path using
- Does not modify inheritance or tags - those are handled separately on PathMetadataEntry when needed.
UpdateFilePathAssociations rebuilds all file-path associations for all files in the package, ensuring that every file is properly linked to its path metadata.
The function performs the following operations:
- Retrieves all files in the package using
ListFiles. - Retrieves all path metadata entries using
GetPathMetadata. - Builds a path metadata map for efficient lookup:
- Creates a map keyed by path strings
- Maps each path to its corresponding PathMetadataEntry pointer
- Associates each file with its path metadata:
- Iterates through all files and their associated paths (
FileEntry.Paths) - For each path in
FileEntry.Paths, extracts the path string and looks up the correspondingPathMetadataEntryin the map - If a matching
PathMetadataEntryexists, callsFileEntry.AssociateWithPathMetadata(pme)to establish the association - This adds the
PathMetadataEntrytoFileEntry.PathMetadataEntries[pathString]and adds theFileEntrytoPathMetadataEntry.AssociatedFileEntries - Processes all paths for each file to handle files with multiple path entries
- Iterates through all files and their associated paths (
- Establishes parent path associations:
- For each
PathMetadataEntry, derives the parent path usingfilepath.DironPathMetadataEntry.Path.Path(after normalizing trailing slashes) - Looks up the parent
PathMetadataEntryin the map - If found, sets
PathMetadataEntry.ParentPathto the parentPathMetadataEntrypointer - This operation never fails - missing parents are valid for root paths or when path metadata is optional
ParentPathis a runtime-only convenience field (not persisted) that remainsnilwhen parents are not found- This builds the path hierarchy needed for inheritance resolution
- For each
This function ensures that the entire package has consistent file-path metadata associations, enabling proper tag inheritance and filesystem property management across all files.
Special metadata files must be saved with specific flags and file types to ensure proper recognition and processing.
- Must use special file types (see File Types System - Special Files)
- Must have reserved file names (e.g.,
__NVPK_PATH_65001__.nvpkpath) - Must support either uncompressed data or LZ4-compressed data with automatic decompression on read
- Must have proper package header flags set
- Type 65000: Package metadata (
__NVPK_PKG_65000__.yaml) - Type 65001: Path metadata (
__NVPK_PATH_65001__.nvpkpath) - Type 65002: Symbolic link metadata (
__NVPK_SYMLINK_65002__.nvpksym) - Type 65003-65535: Reserved for future use
- Bit 6: Has special metadata files (set to 1 when special files exist)
- Bit 5: Has per-file tags (set to 1 if path metadata provides inheritance)
Typefield set to appropriate special file type (65001 for path metadata)CompressionTypeset to 0 (no compression) or 2 (LZ4 compression)EncryptionTypeset to 0x00 (no encryption) - special files should not be encryptedTagsshould includefile_type=special_metadataandmetadata_type=path
Automatic Decompression: When loading path metadata files (via LoadPathMetadataFile or when accessing the FileEntry data), the system automatically decompresses the file data if CompressionType is set to LZ4 (2).
This works similarly to how package-level compression is automatically handled - the decompression occurs transparently when the file data is accessed.
See FileEntry API - Data Management for details on automatic decompression behavior.
This section describes implementation details for path metadata operations.
// SavePathMetadataFile creates and saves the path metadata file
func (p *Package) SavePathMetadataFile(ctx context.Context) error// UpdateSpecialMetadataFlags updates package header flags based on special files
func (p *Package) UpdateSpecialMetadataFlags() errorSavePathMetadataFile creates and saves the path metadata file as a special metadata file in the package.
The function performs the following operations:
- Retrieves the current path metadata entries using
GetPathMetadata. - Marshals the path entries to YAML format, wrapping them in a map with the key
"paths". - Creates a special metadata FileEntry using
CreateSpecialMetadataFilewith:- File type
65001(path metadata) - File name
"__NVPK_PATH_65001__.nvpkpath" - The marshaled YAML data as content
CompressionTypeset to0(none) or2(LZ4) depending on the requested behavior
- File type
- Sets the required tags on the FileEntry:
file_type="special_metadata"(string tag)metadata_type="path"(string tag)format="yaml"(string tag)version=1(integer tag)
- Updates the package header flags by calling
UpdateSpecialMetadataFlags.
LoadPathMetadataFile loads and parses the path metadata file from the package.
The function performs the following operations:
- Locates the path metadata special file (type 65001) in the package.
- Loads the file data using
FileEntry.LoadData(), which automatically decompresses the data ifCompressionTypeis set to LZ4 (2). - Parses the YAML content to extract path metadata entries.
- Updates the package's internal path metadata state with the loaded entries.
The automatic decompression behavior is similar to how package-level compression is handled - when FileEntry.LoadData() is called, it detects the CompressionType field and automatically decompresses the data before returning it.
This ensures that path metadata files compressed with LZ4 are transparently decompressed when loaded, without requiring explicit decompression calls.
See Package Compression API for details on compression and decompression behavior.
UpdateSpecialMetadataFlags updates the package header flags and PackageInfo to reflect the current state of special metadata files, per-file tags, and extended attributes.
- During OpenPackage/Read: Header flags are read from disk and used to populate PackageInfo (header is source of truth)
- During In-Memory Operations: PackageInfo is the source of truth for package state
- During SavePackage/Write: PackageInfo is used to update header flags before writing to disk (PackageInfo is source of truth)
Current Implementation Note: This function currently updates both header flags and PackageInfo simultaneously during in-memory operations. When write operations are fully implemented, the write path will use PackageInfo as the source of truth to update header flags before serialization.
The function:
- Checks for the presence of special metadata files.
- Checks for per-file tags (path metadata properties).
- Checks for extended attributes (filesystem metadata in path metadata entries).
- Updates the package header flags accordingly:
- Sets Bit 6 (FlagHasSpecialMetadata) if special metadata files exist
- Sets Bit 5 (FlagHasPerFileTags) if per-file tags are present
- Sets Bit 3 (FlagHasExtendedAttrs) if extended attributes are present
- Updates the corresponding PackageInfo fields to match header flags.
This ensures the package header and PackageInfo accurately reflect the package structure and capabilities during in-memory operations.
The path association system links FileEntry objects to their corresponding PathMetadataEntry metadata, enabling tag inheritance and filesystem property management.
Note: Since inheritance is now handled only on PathMetadataEntry (not FileEntry), FileEntry no longer maintains ParentPath or InheritedTags properties.
Inheritance is resolved per-path by accessing the associated PathMetadataEntry and walking up its ParentPath chain.
Mode- Unix/Linux path permissions (octal)UID/GID- User and Group IDsACL- Access Control List entriesWindowsAttrs- Windows path attributesExtendedAttrs- Extended attributes mapFlags- Filesystem-specific flags
File-path associations are managed at the struct level, linking FileEntry instances to PathMetadataEntry instances directly.
Associations are established by matching paths between FileEntry.Paths and PathMetadataEntry.Path.
FileEntry Association Methods: See FileEntry API - Path Management for FileEntry.AssociateWithPathMetadata() and FileEntry.GetPathMetadataForPath().
PathMetadataEntry Association Methods: See Path Metadata Structures - PathMetadataEntry Methods for PathMetadataEntry.AssociateWithFileEntry() and PathMetadataEntry.GetAssociatedFileEntries().
Package-Level Association Management: See PathMetadata Management Methods for Package.UpdateFilePathAssociations().
Note: Tag management is performed directly on FileEntry or PathMetadataEntry instances using AddTag(), SetTag(), GetTag(), RemoveTag(), etc. To get inherited or effective tags for a path, use PathMetadataEntry.GetInheritedTags() or PathMetadataEntry.GetEffectiveTags() after retrieving the PathMetadataEntry for the specific path. GetEffectiveTags() includes tags from associated FileEntry instances, treating them as if they were directly applied to the PathMetadataEntry.
This section describes file-path association operations.
This section describes file-path association query methods.
// File-path association query methods (Package-level)
// These methods work with path strings to find and return associated structs
func (p *Package) GetFilePathAssociation(filePath string) (*FilePathAssociation, error)// GetFilesInPath returns all file entries within the specified path.
func (p *Package) GetFilesInPath(path string) ([]*FileEntry, error)// GetPathFiles returns all file entries associated with the specified path.
func (p *Package) GetPathFiles(path string) ([]*FileEntry, error)This section describes path hierarchy analysis methods.
// Path hierarchy analysis
func (p *Package) GetPathTree() (*PathTree, error)// GetPathStats returns statistics for all paths in the package.
func (p *Package) GetPathStats() (map[string]PathStats, error)This section describes symbolic link management methods.
// AddSymlink adds a symbolic link to the package
// Parameters:
// - symlink: SymlinkEntry to add
// Returns:
// - Error if validation fails or symlink cannot be added
// Validation:
// - Calls ValidateSymlinkPaths() to ensure paths are valid and within package root
// - Verifies target exists as FileEntry or PathMetadataEntry directory
// - Returns ErrTypeValidation, ErrTypeSecurity, or ErrTypeNotFound on validation failure
// Returns *PackageError on failure
func (p *Package) AddSymlink(symlink SymlinkEntry) error// Returns *PackageError on failure
func (p *Package) RemoveSymlink(sourcePath string) error// Returns *PackageError on failure
func (p *Package) GetSymlink(sourcePath string) (*SymlinkEntry, error)// Returns *PackageError on failure
func (p *Package) ListSymlinks() ([]SymlinkEntry, error)// Returns *PackageError on failure
func (p *Package) UpdateSymlink(sourcePath string, symlink SymlinkEntry) error// Returns *PackageError on failure
func (p *Package) SaveSymlinkMetadataFile(ctx context.Context, symlink SymlinkEntry) error// Returns *PackageError on failure
func (p *Package) LoadSymlinkMetadataFile(ctx context.Context, fileEntry *FileEntry) (*SymlinkEntry, error)This section describes symlink validation methods.
// ValidateSymlinkPaths validates symlink source and target paths
// Parameters:
// - ctx: Context for cancellation and timeout
// - sourcePath: The symlink source path (where the symlink will be created)
// - targetPath: The symlink target path (where the symlink points to)
// Returns:
// - Error if validation fails
// Validation performed:
// - Both paths are package-relative (start with "/")
// - Both paths are within package root (no ".." escapes)
// - Target path exists as FileEntry or PathMetadataEntry directory
// - Returns ErrTypeValidation for invalid paths
// - Returns ErrTypeSecurity for paths escaping package root
// - Returns ErrTypeNotFound if target does not exist
func (p *Package) ValidateSymlinkPaths(ctx context.Context, sourcePath, targetPath string) error// TargetExists checks if a path exists as FileEntry or directory PathMetadataEntry
// Parameters:
// - ctx: Context for cancellation and timeout
// - path: The path to check (package-relative with leading "/")
// Returns:
// - true if path exists as FileEntry or directory PathMetadataEntry
// - false otherwise
func (p *Package) TargetExists(ctx context.Context, path string) bool// ValidatePathWithinPackageRoot validates that a path is within package root
// Parameters:
// - path: The path to validate (package-relative)
// Returns:
// - Normalized path if valid
// - Error if path escapes package root or is invalid
// - Returns ErrTypeValidation for invalid format
// - Returns ErrTypeSecurity for paths escaping package root
func (p *Package) ValidatePathWithinPackageRoot(path string) (string, error)// PathTree represents the complete path hierarchy
type PathTree struct {
Root *PathNode `json:"root"`
TotalDirs int `json:"total_dirs"`
TotalFiles int `json:"total_files"`
}// PathNode represents a node in the path tree
type PathNode struct {
Path string `json:"path"`
PathMetadata *PathInfo `json:"path_metadata,omitempty"`
Files []*FileEntry `json:"files"`
Children []*PathNode `json:"children"`
}// PathStats provides statistics for a path
type PathStats struct {
FileCount int `json:"file_count"`
TotalSize int64 `json:"total_size"`
CompressedSize int64 `json:"compressed_size"`
LastModified string `json:"last_modified"`
}// SymlinkEntry represents a symbolic link with metadata
type SymlinkEntry struct {
SourcePath string `yaml:"source_path"` // Original symlink path
TargetPath string `yaml:"target_path"` // Target path (resolved)
Properties []Tag `yaml:"properties"` // Symlink-specific tags
Metadata SymlinkMetadata `yaml:"metadata"` // Symlink metadata
FileSystem SymlinkFileSystem `yaml:"filesystem"` // Filesystem properties
}// SymlinkMetadata contains symlink creation and modification information
type SymlinkMetadata struct {
Created time.Time `yaml:"created"` // When symlink was created
Modified time.Time `yaml:"modified"` // When symlink was last modified
Description string `yaml:"description,omitempty"` // Optional description
}// SymlinkFileSystem contains filesystem-specific properties for symlinks
type SymlinkFileSystem struct {
Mode *uint32 `yaml:"mode,omitempty"` // Symlink permissions (octal)
UID *uint32 `yaml:"uid,omitempty"` // User ID
GID *uint32 `yaml:"gid,omitempty"` // Group ID
ACL []ACLEntry `yaml:"acl,omitempty"` // Access Control List
WindowsAttrs *uint32 `yaml:"windows_attrs,omitempty"` // Windows attributes
ExtendedAttrs map[string]string `yaml:"extended_attrs,omitempty"` // Extended attributes
Flags *uint16 `yaml:"flags,omitempty"` // Filesystem-specific flags
}The following methods provide validation for symlink operations:
See 8.5.4.1 Package.ValidateSymlinkPaths Method for the method signature.
Validates that symlink source and target paths are valid and within package boundaries.
- Both paths must be package-relative (start with "/")
- Both paths must be within package root (no ".." escapes)
- Target path must exist as FileEntry or PathMetadataEntry directory
- Returns
ErrTypeValidationfor invalid paths - Returns
ErrTypeSecurityfor paths escaping package root - Returns
ErrTypeNotFoundif target does not exist
See 8.5.4.2 Package.TargetExists Method for the method signature.
Checks if a path exists as FileEntry or directory PathMetadataEntry.
- Verify symlink targets before creating symlinks
- Validate paths during conversion operations
- Check path existence for validation workflows
See 8.5.4.3 Package.ValidatePathWithinPackageRoot Method for the method signature.
Validates that a path is within package root and does not escape boundaries.
- Path must be package-relative format
- No ".." components that escape package root
- Returns normalized path if valid
- Returns
ErrTypeValidationfor invalid format - Returns
ErrTypeSecurityfor paths escaping package root
Symlinks can be created automatically when converting duplicate path entries on a FileEntry to symlinks.
- Select primary path (explicit, custom selector, or lexicographic)
- Create SymlinkEntry for each non-primary path pointing to primary
- Create PathMetadataEntry with
Type: PathMetadataTypeFileSymlink - Update FileEntry to have single primary path
- All symlink paths (source and target) MUST be within package root
- No external filesystem references allowed
- Validation performed via
ValidateSymlinkPaths()before creation
- Symlink targets MUST exist before symlink creation
- Verified via
TargetExists()check - Returns
ErrTypeNotFoundif target does not exist
See File Update API - ConvertPathsToSymlinks for complete conversion API documentation.
// Find FileEntry with multiple paths
entry, _ := pkg.GetFileByPath(ctx, "/app/bin/main")
// Assume entry has paths: ["/app/bin/main", "/usr/local/bin/main", "/opt/main"]
// Convert to symlinks with explicit primary path
options := &SymlinkConvertOptions{
PrimaryPath: Option.Some("/app/bin/main"),
PreservePathMetadata: Option.Some(true),
}
updatedEntry, symlinks, err := pkg.ConvertPathsToSymlinks(ctx, entry, options)
// Result:
// - updatedEntry has single path: "/app/bin/main"
// - symlinks contains 2 SymlinkEntry objects pointing to "/app/bin/main"