Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .devcontainer/post-create.sh
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,18 @@ else
exit 1
fi

# Install mssql_py_core from NuGet (required for bulkcopy functionality)
echo ""
echo "📦 Installing mssql_py_core from NuGet (required for bulkcopy)..."
PYCORE_INSTALLED=false
if bash eng/scripts/install-mssql-py-core.sh; then
echo "✅ mssql_py_core installed successfully"
PYCORE_INSTALLED=true
else
echo "⚠️ mssql_py_core installation failed - bulkcopy functionality will not be available"
echo " You can retry manually: bash eng/scripts/install-mssql-py-core.sh"
fi

# Generate random password for SQL Server
echo ""
echo "Generating SQL Server password..."
Expand Down Expand Up @@ -103,6 +115,11 @@ echo "=============================================="
echo ""
echo "📦 What's ready:"
echo " ✅ C++ extension built"
if [ "$PYCORE_INSTALLED" = "true" ]; then
echo " ✅ mssql_py_core installed (bulkcopy support)"
else
echo " ⚠️ mssql_py_core not installed (bulkcopy unavailable - retry: bash eng/scripts/install-mssql-py-core.sh)"
fi
echo " ✅ SQL Server running (localhost:1433)"
echo " ✅ DB_CONNECTION_STRING set in environment"
echo ""
Expand Down
80 changes: 73 additions & 7 deletions .github/prompts/build-ddbc.prompt.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,11 +113,49 @@ cd mssql_python\pybind && build.bat && cd ..\..

---

## STEP 2: Verify the Build
## STEP 2: Install mssql_py_core from NuGet (Required for Bulkcopy)

The `bulkcopy` method requires the `mssql_py_core` native module, which is distributed as a NuGet package and must be installed separately.

### 2.1 Run the Install Script

**macOS / Linux:**

```bash
# From repository root - downloads and extracts the matching wheel from NuGet
bash eng/scripts/install-mssql-py-core.sh
```

**Windows (PowerShell):**

```powershell
# From repository root
.\eng\scripts\install-mssql-py-core.ps1
```

### 2.2 What the Script Does

1. **Reads** the required version from `eng/versions/mssql-py-core.version`
2. **Downloads** the `mssql-py-core-wheels` NuGet package from the public Azure Artifacts feed
3. **Extracts** the platform-matching wheel (e.g., `mssql_py_core-*-cp313-cp313-linux_x86_64.whl`)
4. **Installs** the `mssql_py_core/` directory at the repository root

### 2.3 Verify Installation

```bash
# From repository root
python -c "import mssql_py_core; print('✅ mssql_py_core loaded:', dir(mssql_py_core))"
```

> ⚠️ **Note:** On Linux build containers with glibc older than 2.34, the import verification is skipped automatically (the `.so` requires glibc 2.34+). The file is still installed correctly.

---

## STEP 3: Verify the Build

**These commands assume you're at the repository root** (which you should be after Step 1).

### 2.1 Check Output File Exists
### 3.1 Check Output File Exists

```bash
# macOS/Linux
Expand All @@ -127,15 +165,15 @@ ls -la mssql_python/ddbc_bindings.*.so
dir mssql_python\ddbc_bindings.*.pyd
```

### 2.2 Verify Import Works
### 3.2 Verify Import Works

```bash
python -c "from mssql_python import connect; print('✅ Import successful')"
```

---

## STEP 3: Clean Build (If Needed)
## STEP 4: Clean Build (If Needed)

If you need a completely fresh build:

Expand Down Expand Up @@ -280,15 +318,43 @@ rm -rf build/
./build.sh
```

### ❌ "No module named 'mssql_py_core'" when using bulkcopy

**Cause:** The `mssql_py_core` NuGet package has not been installed yet.

**Fix:**
```bash
# macOS/Linux - from repository root
bash eng/scripts/install-mssql-py-core.sh

# Windows (PowerShell) - from repository root
.\eng\scripts\install-mssql-py-core.ps1
```

### ❌ "ERROR: No wheel found matching: ..." when running install-mssql-py-core.sh

**Cause:** The NuGet package doesn't contain a wheel for your Python version or platform.

**Fix:**
```bash
# Check the version file
cat eng/versions/mssql-py-core.version

# Check your Python version and platform
python -c "import sys, platform; v=sys.version_info; print(f'cp{v.major}{v.minor}', platform.system(), platform.machine())"

# Contact the team if the combination is unsupported
```

---

## Quick Reference

### One-Liner Build Commands

```bash
# macOS/Linux - Full rebuild from repo root
cd mssql_python/pybind && rm -rf build && ./build.sh && cd ../.. && python -c "from mssql_python import connect; print('✅ Build successful')"
# macOS/Linux - Full rebuild from repo root (includes mssql_py_core)
cd mssql_python/pybind && rm -rf build && ./build.sh && cd ../.. && bash eng/scripts/install-mssql-py-core.sh && python -c "from mssql_python import connect; print('✅ Build successful')"
```

### Build Output Naming Convention
Expand All @@ -305,7 +371,7 @@ cd mssql_python/pybind && rm -rf build && ./build.sh && cd ../.. && python -c "f

## After Building

Once the build succeeds:
Once the build succeeds (Steps 1 and 2 above):

1. **Run tests** → Use `#run-tests`
2. **Test manually** with a connection to SQL Server
Expand Down
44 changes: 33 additions & 11 deletions .github/prompts/run-tests.prompt.md
Original file line number Diff line number Diff line change
Expand Up @@ -247,17 +247,23 @@ tests/test_004_cursor.py:25: DatabaseError

### Test Files and What They Cover

| File | Purpose | Requires DB? |
|------|---------|--------------|
| `test_000_dependencies.py` | Dependency checks | No |
| `test_001_globals.py` | Global state | No |
| `test_002_types.py` | Type conversions | No |
| `test_003_connection.py` | Connection lifecycle | **Yes** |
| `test_004_cursor.py` | Cursor operations | **Yes** |
| `test_005_connection_cursor_lifecycle.py` | Lifecycle management | **Yes** |
| `test_006_exceptions.py` | Error handling | Mixed |
| `test_007_logging.py` | Logging functionality | No |
| `test_008_auth.py` | Authentication | **Yes** |
| File | Purpose | Requires DB? | Requires mssql_py_core? |
|------|---------|--------------|--------------------------|
| `test_000_dependencies.py` | Dependency checks | No | No |
| `test_001_globals.py` | Global state | No | No |
| `test_002_types.py` | Type conversions | No | No |
| `test_003_connection.py` | Connection lifecycle | **Yes** | No |
| `test_004_cursor.py` | Cursor operations | **Yes** | No |
| `test_005_connection_cursor_lifecycle.py` | Lifecycle management | **Yes** | No |
| `test_006_exceptions.py` | Error handling | Mixed | No |
| `test_007_logging.py` | Logging functionality | No | No |
| `test_008_auth.py` | Authentication | **Yes** | No |
| `test_019_bulkcopy.py` | Bulk copy operations | **Yes** | **Yes** |

> ⚠️ **Bulkcopy tests** require `mssql_py_core` to be installed. If not installed, the module is automatically skipped. To install it:
> ```bash
> bash eng/scripts/install-mssql-py-core.sh
> ```

---

Expand Down Expand Up @@ -339,6 +345,22 @@ cd mssql_python/pybind && ./build.sh && cd ../..
python -c "from mssql_python import connect; print('OK')"
```

### ❌ Bulkcopy tests are skipped ("module not available")

**Cause:** `mssql_py_core` is not installed — the bulkcopy tests skip automatically when it can't be imported.

**Fix:**
```bash
# macOS/Linux - from repository root
bash eng/scripts/install-mssql-py-core.sh

# Windows (PowerShell) - from repository root
.\eng\scripts\install-mssql-py-core.ps1

# Verify
python -c "import mssql_py_core; print('✅ mssql_py_core loaded')"
```

### ❌ Tests pass locally but fail in CI

**Cause:** Environment differences (connection string, Python version, OS)
Expand Down
41 changes: 39 additions & 2 deletions .github/prompts/setup-dev-env.prompt.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,13 +134,34 @@ pip install black flake8 autopep8
pip install -e .
```

### 3.5 Verify Python Dependencies
### 3.5 Install mssql_py_core from NuGet (Required for Bulkcopy)

The `bulkcopy` feature requires `mssql_py_core`, a Rust-based native module distributed as a NuGet package. It must be installed separately from pip packages.

**macOS / Linux:**

```bash
# From repository root - downloads and extracts the wheel matching your Python/platform
bash eng/scripts/install-mssql-py-core.sh
```

**Windows (PowerShell):**

```powershell
# From repository root
.\eng\scripts\install-mssql-py-core.ps1
```

> ℹ️ The script reads the version from `eng/versions/mssql-py-core.version`, downloads the NuGet package from the public Azure Artifacts feed, and extracts the `mssql_py_core/` directory to the repository root.

### 3.6 Verify Python Dependencies

```bash
# Check critical packages
python -c "import pybind11; print('✅ pybind11:', pybind11.get_include())"
python -c "import pytest; print('✅ pytest:', pytest.__version__)"
python -c "import mssql_python; print('✅ mssql_python installed')"
python -c "import mssql_py_core; print('✅ mssql_py_core installed (bulkcopy support)')" 2>/dev/null || echo "⚠️ mssql_py_core not installed (bulkcopy unavailable)"
```

---
Expand Down Expand Up @@ -585,6 +606,7 @@ echo "3. Key Packages:" && \
python -c "import pybind11; print(' ✅ pybind11:', pybind11.__version__)" 2>/dev/null || echo " ❌ pybind11 not installed" && \
python -c "import pytest; print(' ✅ pytest:', pytest.__version__)" 2>/dev/null || echo " ❌ pytest not installed" && \
python -c "import mssql_python; print(' ✅ mssql_python installed')" 2>/dev/null || echo " ❌ mssql_python not installed" && \
python -c "import mssql_py_core; print(' ✅ mssql_py_core installed (bulkcopy support)')" 2>/dev/null || echo " ⚠️ mssql_py_core not installed (bulkcopy unavailable - run: bash eng/scripts/install-mssql-py-core.sh)" && \
echo "" && \
echo "4. Build Tools:" && \
cmake --version 2>/dev/null | head -1 | sed 's/^/ ✅ /' || echo " ❌ cmake not found" && \
Expand Down Expand Up @@ -714,6 +736,19 @@ pip install -r requirements.txt -v
pip install <package-name>
```

### ❌ "No module named 'mssql_py_core'" when using bulkcopy

**Cause:** The `mssql_py_core` NuGet package has not been installed yet.

**Fix:**
```bash
# macOS/Linux - from repository root
bash eng/scripts/install-mssql-py-core.sh

# Windows (PowerShell) - from repository root
.\eng\scripts\install-mssql-py-core.ps1
```

### ❌ PowerShell: "Activate.ps1 cannot be loaded because running scripts is disabled"

**Cause:** PowerShell execution policy
Expand Down Expand Up @@ -741,6 +776,7 @@ pip install --upgrade pip && \
pip install -r requirements.txt && \
pip install pybind11 pytest pytest-cov && \
pip install -e . && \
bash eng/scripts/install-mssql-py-core.sh && \
echo "✅ Setup complete!"
```

Expand All @@ -752,14 +788,15 @@ echo "✅ Setup complete!"
| `pytest` | Testing | Running tests |
| `pytest-cov` | Coverage | Coverage reports |
| `azure-identity` | Azure auth | Runtime (in requirements.txt) |
| `mssql_py_core` | Rust/TDS bulkcopy core | Bulkcopy feature (via NuGet script) |

---

## After Setup

Once setup is complete, you can:

1. **Build DDBC extensions** → Use `#build-ddbc`
1. **Build DDBC extensions** → Use `#build-ddbc` (also installs `mssql_py_core` for bulkcopy)
2. **Run tests** → Use `#run-tests`

> 💡 You typically only need to run this setup prompt **once** per machine or after major changes.