Commit Graph

46 Commits

Author SHA1 Message Date
Melchior Reimers
24da2a5861 Added logging, replaced ~25 print() calls, fixed bare except: pass → except (ValueError, IndexError): pass 2026-02-16 11:59:01 +01:00
Melchior Reimers
18801ec27e dded logging, replaced print() and bare except Exception: continue with except (ValueError, KeyError) + debug log 2026-02-16 11:58:46 +01:00
Melchior Reimers
71f8614cb5 Crash bug: self.questdb_url → self.db_url, self.auth → DB_AUTH in delete_analytics_for_date()
Table name validation in get_existing_dates() and delete_analytics_for_date()
Exchange validation in calculate_custom_analytics_daily()
2026-02-16 11:58:14 +01:00
Melchior Reimers
846f5e76fe centralized input validation with 8 functions: validate_isin, validate_exchange, validate_date, validate_int_range, sanitize_sql_string, validate_isin_list, validate_exchange_list, validate_table_name 2026-02-16 11:57:19 +01:00
Melchior Reimers
7c802a3907 empty package init 2026-02-16 11:56:55 +01:00
Melchior Reimers
1dc79b8b64 Refactor: Code-Qualität verbessert und Projektstruktur aufgeräumt
Some checks failed
Deployment / deploy-docker (push) Has been cancelled
- daemon.py: gc.collect() entfernt, robustes Scheduling (last_run_date statt Minuten-Check),
  Exchange Registry Pattern eingeführt (STREAMING_EXCHANGES/STANDARD_EXCHANGES)
- deutsche_boerse.py: Thread-safe User-Agent Rotation bei Rate-Limits,
  Logging statt print(), Feiertags-Prüfung, aufgeteilte Parse-Methoden
- eix.py: Logging statt print(), spezifische Exception-Typen statt blankem except
- read.py gelöscht und durch scripts/inspect_gzip.py ersetzt (Streaming-basiert)
- Utility-Scripts in scripts/ verschoben (cleanup_duplicates, restore_and_fix, verify_fix)
2026-02-01 08:18:55 +01:00
Melchior Reimers
cf55a0bd06 Fix: Analytics Worker berechnet heute/gestern IMMER neu
Some checks failed
Deployment / deploy-docker (push) Has been cancelled
- Neue force_recalculate_date() Methode löscht alte Daten vor Neuberechnung
- Heute und gestern werden bei jedem Stunden-Check neu berechnet
- Behebt Problem, dass neue Trades nicht in Analytics aufgenommen wurden
2026-01-29 22:36:22 +01:00
Melchior Reimers
9cd84e0855 Fix: Streaming-Verarbeitung für EIX um RAM-Überlauf zu verhindern
Some checks failed
Deployment / deploy-docker (push) Has been cancelled
- EIX verarbeitet jetzt eine Datei nach der anderen (nicht alle auf einmal)
- Speicher wird nach jeder Datei freigegeben (gc.collect)
- Day-basiertes Caching für Duplikatprüfung mit Cache-Clearing
- Reduziert RAM-Verbrauch von 8GB+ auf unter 500MB
2026-01-29 16:17:11 +01:00
Melchior Reimers
a21e036bb4 Add Börsenag exchanges (DUSA, DUSB, DUSC, DUSD, HAMA, HAMB, HANA, HANB)
- New boersenag.py with support for Düsseldorf, Hamburg, and Hannover exchanges
- Proper .gitignore to exclude venv and temp files
2026-01-29 16:08:14 +01:00
Melchior Reimers
a07319d957 Fix: Analytics Worker berechnet jetzt alle Tabellen pro Tag
Some checks failed
Deployment / deploy-docker (push) Has been cancelled
2026-01-29 16:00:09 +01:00
Melchior Reimers
79a80141e9 Fix: Analytics Worker berechnet jetzt alle Tabellen pro Tag
Some checks failed
Deployment / deploy-docker (push) Has been cancelled
2026-01-29 14:53:57 +01:00
Melchior Reimers
77651d64c1 Fix: Analytics Worker berechnet jetzt alle Tabellen pro Tag
All checks were successful
Deployment / deploy-docker (push) Successful in 18s
2026-01-27 14:20:27 +01:00
Melchior Reimers
459c24fcd3 Fix: Analytics Worker berechnet jetzt alle Tabellen pro Tag
All checks were successful
Deployment / deploy-docker (push) Successful in 17s
2026-01-27 14:12:26 +01:00
Melchior Reimers
4fd93541a2 Fix: Analytics Worker berechnet jetzt alle Tabellen pro Tag
All checks were successful
Deployment / deploy-docker (push) Successful in 16s
2026-01-27 13:57:07 +01:00
Melchior Reimers
d66f261bcc updated dashboard
All checks were successful
Deployment / deploy-docker (push) Successful in 19s
2026-01-27 13:41:31 +01:00
Melchior Reimers
f626766523 updated dashboard
All checks were successful
Deployment / deploy-docker (push) Successful in 17s
2026-01-27 13:36:28 +01:00
Melchior Reimers
f3a59ac778 updated dashboard
All checks were successful
Deployment / deploy-docker (push) Successful in 18s
2026-01-27 13:08:14 +01:00
Melchior Reimers
996cf0b1a1 updated dashboard
All checks were successful
Deployment / deploy-docker (push) Successful in 18s
2026-01-27 13:07:01 +01:00
Melchior Reimers
01f4c23f51 updated dashboard
All checks were successful
Deployment / deploy-docker (push) Successful in 16s
2026-01-27 11:30:50 +01:00
Melchior Reimers
228212cbab updated dashboard
All checks were successful
Deployment / deploy-docker (push) Successful in 17s
2026-01-27 11:09:52 +01:00
Melchior Reimers
b25bab2288 updated dashboard
All checks were successful
Deployment / deploy-docker (push) Successful in 18s
2026-01-27 11:00:55 +01:00
Melchior Reimers
390f2d91a4 updated dashboard
All checks were successful
Deployment / deploy-docker (push) Successful in 18s
2026-01-27 10:56:56 +01:00
Melchior Reimers
22b09669c1 updated dashboard
All checks were successful
Deployment / deploy-docker (push) Successful in 18s
2026-01-27 10:48:11 +01:00
Melchior Reimers
e124f38ac9 updated dashboard
All checks were successful
Deployment / deploy-docker (push) Successful in 18s
2026-01-27 10:38:29 +01:00
Melchior Reimers
e71d1a061e fixed deutsche boerse
All checks were successful
Deployment / deploy-docker (push) Successful in 17s
2026-01-27 10:14:27 +01:00
Melchior Reimers
44e7004fc9 added more exchanges
All checks were successful
Deployment / deploy-docker (push) Successful in 40s
2026-01-27 09:59:43 +01:00
Melchior Reimers
66c472be05 updated
All checks were successful
Deployment / deploy-docker (push) Successful in 16s
2026-01-25 18:38:37 +01:00
Melchior Reimers
df478b0ea4 updated
All checks were successful
Deployment / deploy-docker (push) Successful in 16s
2026-01-25 18:31:13 +01:00
Melchior Reimers
9185ec3ef6 updated
All checks were successful
Deployment / deploy-docker (push) Successful in 17s
2026-01-25 18:07:57 +01:00
Melchior Reimers
b4b96b96dc updated
All checks were successful
Deployment / deploy-docker (push) Successful in 16s
2026-01-25 18:02:20 +01:00
Melchior Reimers
eb9c822cff updated
All checks were successful
Deployment / deploy-docker (push) Successful in 16s
2026-01-25 17:39:15 +01:00
Melchior Reimers
33f5c90fce updated
All checks were successful
Deployment / deploy-docker (push) Successful in 17s
2026-01-25 17:36:29 +01:00
Melchior Reimers
786fef2e71 performance improvements by pre-defining queries
All checks were successful
Deployment / deploy-docker (push) Successful in 17s
2026-01-25 17:11:05 +01:00
Melchior Reimers
bbc5ec76d1 fixing dashboard
All checks were successful
Deployment / deploy-docker (push) Successful in 16s
2026-01-25 16:51:35 +01:00
Melchior Reimers
b9062c5dac now downloads historical eix dat
All checks were successful
Deployment / deploy-docker (push) Successful in 14s
2026-01-25 16:44:43 +01:00
Melchior Reimers
2ef8a8bf2f fixing dashboard
All checks were successful
Deployment / deploy-docker (push) Successful in 15s
2026-01-25 15:14:13 +01:00
Melchior Reimers
d764a0b8ce fixing dashboard
All checks were successful
Deployment / deploy-docker (push) Successful in 13s
2026-01-25 15:11:52 +01:00
Melchior Reimers
9161300cfc Fix empty name field error in metadata saving
All checks were successful
Deployment / deploy-docker (push) Successful in 15s
2026-01-25 15:07:17 +01:00
Melchior Reimers
445eb0b482 fixing dashboard
All checks were successful
Deployment / deploy-docker (push) Successful in 38s
2026-01-25 15:00:53 +01:00
Melchior Reimers
dbd4fbfb47 fix: date parsing, continent grouping, dynamic analytics, and sector scraping
All checks were successful
Deployment / deploy-docker (push) Successful in 5s
2026-01-23 18:17:02 +01:00
Melchior Reimers
1086c4aa1d fix: EIX scraping, LS endpoints, and premium Dashboard analytics
All checks were successful
Deployment / deploy-docker (push) Successful in 10s
2026-01-23 18:09:00 +01:00
Melchior Reimers
b0707443e8 fix: resolve SyntaxError in ls.py due to indentation
All checks were successful
Deployment / deploy-docker (push) Successful in 3s
2026-01-23 17:59:22 +01:00
Melchior Reimers
608af97a93 feat: add web dashboard and metadata enrichment daemon
All checks were successful
Deployment / deploy-docker (push) Successful in 4s
2026-01-23 17:49:48 +01:00
Melchior Reimers
47a8ceab32 feat: add automatic historical fetch on startup if DB is empty
All checks were successful
Deployment / deploy-docker (push) Successful in 4s
2026-01-23 17:44:06 +01:00
Melchior Reimers
746c7167b0 feat: move daemon to docker with daily schedule and deduplication
All checks were successful
Deployment / deploy-docker (push) Successful in 17s
2026-01-23 17:24:05 +01:00
Melchior Reimers
ccfee645e8 first commit 2026-01-23 16:30:35 +01:00