All,
I'm having a problem with a server application that uses a
Django
model to write data to MySQL.
The original setup was using Twisted to gather messages
using various
protocols. Some data from those messages was written to a
MySQL
database using twisted.enterprise.adbapi. There was also a
statistical
reporting web application written in PHP that aggregated
info from the
db.
I rewrote and expanded the web app using Django this summer.
When time
came to make some large changes to the code which writes to
the db, I
decided to try to replace the hand-coded sql and
twisted.enterprise.adbapi, with code that used the Django
model
(DRY).
I rewrote all the database writes to use the Django model
for the web
app, and wrapped the whole procedure in twisted
callInThread. Later,
although we hadn't seen problems in test (the test server
doesn't get
the load of the real one) I back-ported changes similar to
http://code
.djangoproject.com/ticket/5632 to 0.96. This all worked
fine during dev and test, for about 3 months.
When we moved from test to beta, we started getting this
error:
_mysql_exceptions.InterfaceError: (0, '')
It happens every 4-5 days and we have to bounce the twistd
reactor to
get stuff working again. The database in question averages
425,000
queries a day. I don't have a formal breakdown on what's
coming from
where, but my best napkin calculation is that around 80% of
those come
from this back-end. Since this is an internal use app only,
and in
beta, we run this app Debug=True in Django settings. The
whole/box app
doesn't seem to run out of memory, but we are running a lot
more
queries in beta, and after reading this:
http://groups.
google.com/group/django-users/browse_thread/thread/a32b31a2e
9fb3815/76b595946f0e02b7
about debug query saving causing a similar error, I added a
call to
db.reset_queries(), which did not help.
I also pulled the database calls out of callInThread for
debugging
purposes, and went back to stock 0.96. So the code that uses
Django
from twisted is now running in the main thread. Still seeing
the
problem.
Note that while this is happening from the back-end Twisted
app, the
actual web application, running on Apache/mod_python
continues to
connect without problems. We're allowing up to 100
connections on
mysql, but have only seen between 6-12 actual when this
happens.
Any help on where I might start to look next or what to try
would be
appreciated.
Regards,
Brian
Environment
Red Hat Enterprise 4.
Python 2.3.4 (version that is stock with RHLE 4).
MySQL 5.0.27 (RPM from MySQL AB for RHEL 4)
MySQLdb 1.2.1p2 -- compiled on this box against the libs
from 5.0.27
Django 0.96 release.
Twisted 2.5.0
Stack trace:
Traceback (most recent call last):
File
"/usr/lib/python2.3/site-packages/vertis/dart2server/
dartxmlreader.py", line 41, in connectionLost
self.factory.processData(self.dartTree)
File "Dart2Server.tac", line 69, in
processData
self.service.processData(theelementtree)
File "Dart2Server.tac", line 146, in
processData
d = dartPlugin.storeData(dataobject)
File
"/usr/lib/python2.3/site-packages/vertis/dart2server/
plugins/darttwisttodb.py", line 301, in storeData
result = self._errorWrap(dataobj)
--- <exception caught here> ---
File
"/usr/lib/python2.3/site-packages/vertis/dart2server/
plugins/darttwisttodb.py", line 47, in _errorWrap
return self._realStoreTWiSTJobData(dataobj)
File
"/usr/lib/python2.3/site-packages/vertis/dart2server/
plugins/darttwisttodb.py", line 218, in
_realStoreTWiSTJobData
clientCode = self.dartGetOrCreate(tRoot,{'.//
ClientCode':'client_code'},'Client',defval='XXXX')
File
"/usr/lib/python2.3/site-packages/vertis/dart2server/
plugins/darttwisttodb.py", line 96, in dartGetOrCreate
model_obj,created =
dart_model.objects.get_or_create(**key_dict)
File
"/usr/lib/python2.3/site-packages/django/db/models/
manager.py", line 76, in get_or_create
return
self.get_query_set().get_or_create(**kwargs)
File
"/usr/lib/python2.3/site-packages/django/db/models/
query.py", line 274, in get_or_create
return self.get(**kwargs), False
File
"/usr/lib/python2.3/site-packages/django/db/models/
query.py", line 250, in get
obj_list = list(clone)
File
"/usr/lib/python2.3/site-packages/django/db/models/
query.py", line 108, in __iter__
return iter(self._get_data())
File
"/usr/lib/python2.3/site-packages/django/db/models/
query.py", line 470, in _get_data
self._result_cache = list(self.iterator())
File
"/usr/lib/python2.3/site-packages/django/db/models/
query.py", line 182, in iterator
cursor = connection.cursor()
File
"/usr/lib/python2.3/site-packages/django/db/backends/
mysql/base.py", line 80, in cursor
if not self._valid_connection():
File
"/usr/lib/python2.3/site-packages/django/db/backends/
mysql/base.py", line 70, in _valid_connection
self.connection.ping()
_mysql_exceptions.InterfaceError: (0, '')
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "Django users" group.
To post to this group, send email to django-users googlegroups.com
To unsubscribe from this group, send email to
django-users-unsubscribegooglegroups.com
For more options, visit this group at htt
p://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---
|