1.2.4 Performance Issue

We are having a performance issue where getting a group listing is taking an inordinate amount of time (~ 10 seconds) for a set of 120 or so groups, with one group having over 7000 members and a few others with 700 members.  We have set up the LDAP query to only return the group name (RDN).  However, Penrose appears to construct the entire entry in response to this query, which takes a long time (it parses on the order of 16000 rows). Our database reports that returning the data from the SQL query takes less than a second, so we don’t think it’s a database problem.  

 

This query is rather common in our production scenario, so I’d like to find a way to speed this up, whether by preventing Penrose from creating the whole entry or some kind of (memory) caching.  Persistent caching is not an option, and since the database doesn̵7;t seem to be the bottleneck anyway, doesn̵7;t seem that useful.  Group memberships are changed daily, and I’ve been told not to cache for too long (think minutes) because one of the problems Penrose solves for us is (near-)real-time updates.  Any ideas?

 

My group mapping looks something like this:

 

<entry dn=”gidNumber=&#8230;,ou=groups, dc=blah>;

 &nbsp; &nbsp; &nbsp; &nbsp;   ; <oc>groupOfNames</oc>;

 &nbsp; &nbsp; &nbsp; &nbsp;   ; <at name=”gidNumber221;, rid=”;true”;>

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ;  <variable>ug.GROUPNAME</variable&gt;

 &nbsp; &nbsp; &nbsp; &nbsp;   ; </at>;

 &nbsp; &nbsp; &nbsp; &nbsp;   ; <at name=”cn”>;

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ;  <expression>return ug.GROUPNAME + “ Wiki Group̶1;</expression>

 &nbsp; &nbsp; &nbsp; &nbsp;   ; </at>;

 &nbsp; &nbsp; &nbsp; &nbsp;   ; <at name=”description&#8221;>

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ;  <expression>221;The group for the wiki called “ + ug.GROUPNAME</expression>

 &nbsp; &nbsp; &nbsp; &nbsp;   ; </at>;

 &nbsp; &nbsp; &nbsp; &nbsp;   ; <at name=”member”;>

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ;  <expression foreach=”ug.EMAILR21; variable = “email”&gt;

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; return &#8220;mail=&#8221; + email + “, ou=users, dc=blah221;;

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ;  </expression>

 &nbsp; &nbsp; &nbsp; &nbsp;   ; </at>;

 

 &nbsp; &nbsp; &nbsp; &nbsp;   ; <source name=R21;ug”;>

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ;  <source-name>ug</source-name>

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ;  <field name=̶1;GROUPNAME”>

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; <variable>gidNumber</variable>

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ;  </field>

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ;  <field name=̶1;EMAILR21;>

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; <expression foreach=”member”; var=”;um”&gt;

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp;  int i = um.indexOf(“=&#8221;);

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp;  int j = um.indexOf(“,&#8221;, i+1);

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp;  if(i > 0 and j > 0)

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; return um.substring(i+1, j);

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp;  else

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; return um;

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; </expression>

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ;  </field>

 &nbsp; &nbsp; &nbsp; &nbsp;   ; </source>

</entry>

 

The source “ug&#8221; looks like this:

 

<source name=̶1;ug”>

 &nbsp; &nbsp; &nbsp; &nbsp;   ; <connection-name&gt;MyConnection</connection-name>

 &nbsp; &nbsp; &nbsp; &nbsp;   ; <field name=̶1;EMAILR21; type=̶1;CHAR̶1; primaryKey=”true” length=221;60̶1;/>

 &nbsp; &nbsp; &nbsp; &nbsp;   ; <field name=̶1;GROUPNAME” type=̶1;CHAR̶1; primaryKey=”true” length=̶1;”/&gt;

 &nbsp; &nbsp; &nbsp; &nbsp;   ; <parameter>

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ;  <param-name>loadingMethod&lt;/param-name>

 &nbsp; &nbsp; &nbsp; &nbsp;   ; &nbsp; &nbsp; &nbsp; &nbsp;   ;  <param-value>loadAll</param-value&gt;

 &nbsp; &nbsp; &nbsp; &nbsp;   ; </parameter>

</source>

 

The SQL generated from this on the group query is quite straightforward:

 

SELECT DISTINCT EMAIL, GROUPNAME FROM UG where GROUPNAME is not NULL order by GROUPNAME, EMAIL

 

Thanks,

Rich