product
This commit is contained in:
237
QUICK_START_LOCAL.md
Normal file
237
QUICK_START_LOCAL.md
Normal file
@@ -0,0 +1,237 @@
|
||||
# Quick Start - Testing Locally Without Docker
|
||||
|
||||
## Option 1: Basic Logic Testing (Easiest - No Dependencies)
|
||||
|
||||
```bash
|
||||
# Run the local test script to verify all core logic works
|
||||
python test_local.py
|
||||
```
|
||||
|
||||
**Expected Output:**
|
||||
```
|
||||
✓ Date conversion working
|
||||
✓ TXNIND calculation working
|
||||
✓ ACH Parser: Extracted 178 transactions
|
||||
✓ Filename parsing working
|
||||
✓ Configuration loaded correctly
|
||||
✓ ALL TESTS PASSED
|
||||
```
|
||||
|
||||
**What This Tests:**
|
||||
- ✓ Data transformations (dates, amounts, indicators)
|
||||
- ✓ ACH file parsing (178 transactions)
|
||||
- ✓ Field mapping logic
|
||||
- ✓ Configuration loading
|
||||
- ✗ SFTP (not included)
|
||||
- ✗ Database (not included)
|
||||
|
||||
**Time:** ~2 seconds
|
||||
**Dependencies:** None (uses only Python stdlib + existing ach_parser)
|
||||
|
||||
---
|
||||
|
||||
## Option 2: Unit Tests
|
||||
|
||||
```bash
|
||||
# Install pytest if not already done
|
||||
pip install pytest
|
||||
|
||||
# Run unit tests
|
||||
pytest tests/ -v
|
||||
```
|
||||
|
||||
**What This Tests:**
|
||||
- ✓ Date conversion edge cases
|
||||
- ✓ TXNIND calculation for positive/negative amounts
|
||||
- ✓ Amount conversion
|
||||
- ✓ Transaction mapping
|
||||
- ✓ Filename parsing (valid and invalid)
|
||||
- ✓ Proper error handling
|
||||
|
||||
**Time:** ~5 seconds
|
||||
|
||||
---
|
||||
|
||||
## Option 3: Mock SFTP Server (No Docker)
|
||||
|
||||
### Start the SFTP Server
|
||||
|
||||
```bash
|
||||
# Terminal 1: Start mock SFTP server
|
||||
python tests/mock_sftp_server.py
|
||||
```
|
||||
|
||||
Expected output:
|
||||
```
|
||||
Mock SFTP server listening on 127.0.0.1:2222
|
||||
Username: ipks, Password: ipks_password
|
||||
Server running. Press CTRL+C to stop.
|
||||
```
|
||||
|
||||
### Test SFTP Connection
|
||||
|
||||
```bash
|
||||
# Terminal 2: Test SFTP connection
|
||||
sftp -P 2222 ipks@127.0.0.1
|
||||
# Password: ipks_password
|
||||
# Commands:
|
||||
# ls
|
||||
# cd HDFC/NACH
|
||||
# put ACH_99944_19012026103217_001.txt
|
||||
# quit
|
||||
```
|
||||
|
||||
### Configure for Testing
|
||||
|
||||
Edit `.env`:
|
||||
```
|
||||
SFTP_HOST=127.0.0.1
|
||||
SFTP_PORT=2222
|
||||
SFTP_USERNAME=ipks
|
||||
SFTP_PASSWORD=ipks_password
|
||||
POLL_INTERVAL_MINUTES=1
|
||||
```
|
||||
|
||||
### Copy Test Files to Mock SFTP
|
||||
|
||||
```bash
|
||||
# Terminal 3: Setup test files
|
||||
mkdir -p sftp_data/HDFC/NACH
|
||||
cp ACH_99944_19012026103217_001.txt sftp_data/HDFC/NACH/
|
||||
```
|
||||
|
||||
### Run Application with Mock SFTP
|
||||
|
||||
```bash
|
||||
# Terminal 4: Run application
|
||||
# (Will fail on database but shows SFTP working)
|
||||
python main.py
|
||||
```
|
||||
|
||||
**What This Tests:**
|
||||
- ✓ SFTP connection
|
||||
- ✓ File discovery
|
||||
- ✓ File download to local staging
|
||||
- ✓ ACH parsing
|
||||
- ✗ Database insertion (will fail - no Oracle)
|
||||
|
||||
**Time:** 30+ seconds per cycle
|
||||
**Dependencies:** paramiko, cryptography
|
||||
|
||||
---
|
||||
|
||||
## Summary Table
|
||||
|
||||
| Test Method | Setup Time | Run Time | Tests SFTP | Tests DB | Difficulty |
|
||||
|---|---|---|---|---|---|
|
||||
| Basic Logic | <1 min | ~2s | ✗ | ✗ | Easy |
|
||||
| Unit Tests | 1 min | ~5s | ✗ | ✗ | Easy |
|
||||
| Mock SFTP | 2 min | 30s+ | ✓ | ✗ | Medium |
|
||||
| With Oracle | 15+ min | 1-2 min | ✓ | ✓ | Hard |
|
||||
|
||||
---
|
||||
|
||||
## Recommended Testing Path
|
||||
|
||||
**Step 1: Verify Core Logic (2 seconds)**
|
||||
```bash
|
||||
python test_local.py
|
||||
```
|
||||
✓ Confirms data transformation, parsing, and configuration work
|
||||
|
||||
**Step 2: Run Unit Tests (5 seconds)**
|
||||
```bash
|
||||
pytest tests/ -v
|
||||
```
|
||||
✓ Confirms edge cases and error handling
|
||||
|
||||
**Step 3: Test SFTP Without Docker (30+ seconds)**
|
||||
```bash
|
||||
# Terminal 1
|
||||
python tests/mock_sftp_server.py
|
||||
|
||||
# Terminal 2 (when ready to test)
|
||||
python main.py
|
||||
# Will fail on DB but shows SFTP works
|
||||
```
|
||||
✓ Confirms SFTP file operations work
|
||||
|
||||
**Step 4: Full Integration (when you have Oracle)**
|
||||
- Install Oracle Instant Client
|
||||
- Create database tables
|
||||
- Update .env with real credentials
|
||||
- Run `python main.py` for full pipeline
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "ImportError: No module named 'paramiko'"
|
||||
Only needed for Option 3 (mock SFTP).
|
||||
```bash
|
||||
pip install paramiko cryptography
|
||||
```
|
||||
|
||||
### "Address already in use" on port 2222
|
||||
Wait 30 seconds or use different port:
|
||||
```bash
|
||||
# Edit tests/mock_sftp_server.py:
|
||||
start_mock_sftp_server(port=2223)
|
||||
```
|
||||
|
||||
### Test data not found
|
||||
Create test files:
|
||||
```bash
|
||||
mkdir -p sftp_data/HDFC/NACH
|
||||
cp ACH_99944_19012026103217_001.txt sftp_data/HDFC/NACH/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## What You Can Test WITHOUT Docker
|
||||
|
||||
✓ All data transformation logic (100%)
|
||||
✓ ACH file parsing (100%)
|
||||
✓ Configuration loading (100%)
|
||||
✓ Filename parsing (100%)
|
||||
✓ SFTP operations (with mock server)
|
||||
✓ Unit tests (100%)
|
||||
|
||||
## What Still Requires Oracle
|
||||
|
||||
✗ Database insertion
|
||||
✗ Duplicate detection (stores in DB)
|
||||
✗ Full pipeline end-to-end
|
||||
✗ Production deployment
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Now**: Run `python test_local.py` to verify everything works locally
|
||||
2. **Next**: Read `LOCAL_TESTING.md` for more detailed testing options
|
||||
3. **Then**: When ready, follow `SETUP.md` to set up with Oracle database
|
||||
4. **Finally**: Deploy to production following `DEPLOYMENT.md`
|
||||
|
||||
---
|
||||
|
||||
## Key Files for Local Testing
|
||||
|
||||
- `test_local.py` - Quick verification script (run first)
|
||||
- `LOCAL_TESTING.md` - Detailed testing guide
|
||||
- `tests/test_*.py` - Unit tests
|
||||
- `tests/mock_sftp_server.py` - Python-based SFTP server (no Docker needed)
|
||||
- `.env` - Configuration file
|
||||
|
||||
---
|
||||
|
||||
## No Docker? No Problem!
|
||||
|
||||
All the core processing logic can be tested locally without Docker:
|
||||
- ✓ Data transformations
|
||||
- ✓ File parsing
|
||||
- ✓ Field mapping
|
||||
- ✓ Configuration
|
||||
- ✓ Basic SFTP (with mock server)
|
||||
|
||||
Only the database integration requires Oracle to be installed, which is a one-time setup.
|
||||
Reference in New Issue
Block a user