This commit is contained in:
2026-02-02 13:06:07 +05:30
commit 1b173f992a
41 changed files with 9380 additions and 0 deletions

237
QUICK_START_LOCAL.md Normal file
View File

@@ -0,0 +1,237 @@
# Quick Start - Testing Locally Without Docker
## Option 1: Basic Logic Testing (Easiest - No Dependencies)
```bash
# Run the local test script to verify all core logic works
python test_local.py
```
**Expected Output:**
```
✓ Date conversion working
✓ TXNIND calculation working
✓ ACH Parser: Extracted 178 transactions
✓ Filename parsing working
✓ Configuration loaded correctly
✓ ALL TESTS PASSED
```
**What This Tests:**
- ✓ Data transformations (dates, amounts, indicators)
- ✓ ACH file parsing (178 transactions)
- ✓ Field mapping logic
- ✓ Configuration loading
- ✗ SFTP (not included)
- ✗ Database (not included)
**Time:** ~2 seconds
**Dependencies:** None (uses only Python stdlib + existing ach_parser)
---
## Option 2: Unit Tests
```bash
# Install pytest if not already done
pip install pytest
# Run unit tests
pytest tests/ -v
```
**What This Tests:**
- ✓ Date conversion edge cases
- ✓ TXNIND calculation for positive/negative amounts
- ✓ Amount conversion
- ✓ Transaction mapping
- ✓ Filename parsing (valid and invalid)
- ✓ Proper error handling
**Time:** ~5 seconds
---
## Option 3: Mock SFTP Server (No Docker)
### Start the SFTP Server
```bash
# Terminal 1: Start mock SFTP server
python tests/mock_sftp_server.py
```
Expected output:
```
Mock SFTP server listening on 127.0.0.1:2222
Username: ipks, Password: ipks_password
Server running. Press CTRL+C to stop.
```
### Test SFTP Connection
```bash
# Terminal 2: Test SFTP connection
sftp -P 2222 ipks@127.0.0.1
# Password: ipks_password
# Commands:
# ls
# cd HDFC/NACH
# put ACH_99944_19012026103217_001.txt
# quit
```
### Configure for Testing
Edit `.env`:
```
SFTP_HOST=127.0.0.1
SFTP_PORT=2222
SFTP_USERNAME=ipks
SFTP_PASSWORD=ipks_password
POLL_INTERVAL_MINUTES=1
```
### Copy Test Files to Mock SFTP
```bash
# Terminal 3: Setup test files
mkdir -p sftp_data/HDFC/NACH
cp ACH_99944_19012026103217_001.txt sftp_data/HDFC/NACH/
```
### Run Application with Mock SFTP
```bash
# Terminal 4: Run application
# (Will fail on database but shows SFTP working)
python main.py
```
**What This Tests:**
- ✓ SFTP connection
- ✓ File discovery
- ✓ File download to local staging
- ✓ ACH parsing
- ✗ Database insertion (will fail - no Oracle)
**Time:** 30+ seconds per cycle
**Dependencies:** paramiko, cryptography
---
## Summary Table
| Test Method | Setup Time | Run Time | Tests SFTP | Tests DB | Difficulty |
|---|---|---|---|---|---|
| Basic Logic | <1 min | ~2s | ✗ | ✗ | Easy |
| Unit Tests | 1 min | ~5s | ✗ | ✗ | Easy |
| Mock SFTP | 2 min | 30s+ | ✓ | ✗ | Medium |
| With Oracle | 15+ min | 1-2 min | ✓ | ✓ | Hard |
---
## Recommended Testing Path
**Step 1: Verify Core Logic (2 seconds)**
```bash
python test_local.py
```
✓ Confirms data transformation, parsing, and configuration work
**Step 2: Run Unit Tests (5 seconds)**
```bash
pytest tests/ -v
```
✓ Confirms edge cases and error handling
**Step 3: Test SFTP Without Docker (30+ seconds)**
```bash
# Terminal 1
python tests/mock_sftp_server.py
# Terminal 2 (when ready to test)
python main.py
# Will fail on DB but shows SFTP works
```
✓ Confirms SFTP file operations work
**Step 4: Full Integration (when you have Oracle)**
- Install Oracle Instant Client
- Create database tables
- Update .env with real credentials
- Run `python main.py` for full pipeline
---
## Troubleshooting
### "ImportError: No module named 'paramiko'"
Only needed for Option 3 (mock SFTP).
```bash
pip install paramiko cryptography
```
### "Address already in use" on port 2222
Wait 30 seconds or use different port:
```bash
# Edit tests/mock_sftp_server.py:
start_mock_sftp_server(port=2223)
```
### Test data not found
Create test files:
```bash
mkdir -p sftp_data/HDFC/NACH
cp ACH_99944_19012026103217_001.txt sftp_data/HDFC/NACH/
```
---
## What You Can Test WITHOUT Docker
✓ All data transformation logic (100%)
✓ ACH file parsing (100%)
✓ Configuration loading (100%)
✓ Filename parsing (100%)
✓ SFTP operations (with mock server)
✓ Unit tests (100%)
## What Still Requires Oracle
✗ Database insertion
✗ Duplicate detection (stores in DB)
✗ Full pipeline end-to-end
✗ Production deployment
---
## Next Steps
1. **Now**: Run `python test_local.py` to verify everything works locally
2. **Next**: Read `LOCAL_TESTING.md` for more detailed testing options
3. **Then**: When ready, follow `SETUP.md` to set up with Oracle database
4. **Finally**: Deploy to production following `DEPLOYMENT.md`
---
## Key Files for Local Testing
- `test_local.py` - Quick verification script (run first)
- `LOCAL_TESTING.md` - Detailed testing guide
- `tests/test_*.py` - Unit tests
- `tests/mock_sftp_server.py` - Python-based SFTP server (no Docker needed)
- `.env` - Configuration file
---
## No Docker? No Problem!
All the core processing logic can be tested locally without Docker:
- ✓ Data transformations
- ✓ File parsing
- ✓ Field mapping
- ✓ Configuration
- ✓ Basic SFTP (with mock server)
Only the database integration requires Oracle to be installed, which is a one-time setup.