Pete Muir wrote on Mar 28, 2008 07:50 PM:
It's possible, there is an open feature request for enabling caching which might help.

Hmm, OK. But I don't know if this would help. If I understand aright, then UserTransaction contains all the entities catched by the executed SQL queries?!? Therefore EntityConverter has to (NamingContext).lookup all those entities. So I would suggest another strategy: After, e. g. 10, conversions in series of the same type of entity, EntityConverter could (if it's possible) catch all entities of this type and push them into it's cache. Introducing a property to turn this strategy on/off and a property to specify the number of conversions of a entity type in series needed to catch all entities, this wouldn't affect current applications and allow a fine-grained tuning. But as I said: if I understand aright ;)

Pete Muir wrote on Mar 28, 2008 07:50 PM:
Changing a name won't help. Can you post the stack trace for calls to lookup the UserTransaction, if that it is where it is slow.

Of course I'll provide you with all information you need, if I can. That's where I've to ask for a file upload ;)

Here is a Call Tree and a Method List. I could provide the snapshot at all, but I don't know if you need it and if you use the profiler I'm using (for OS projects it's free).

Jens Weintraut wrote on Mar 31, 2008 09:24 AM:
Hmm, OK. But I don't know if this would help. If I understand aright, then UserTransaction contains all the entities catched by the executed SQL queries?!?

I don't think so. Do you mean the persistence context?

Therefore EntityConverter has to (NamingContext).lookup all those entities. So I would suggest another strategy: After, e. g. 10, conversions in series of the same type of entity, EntityConverter could (if it's possible) catch all entities of this type and push them into it's cache. Introducing a property to turn this strategy on/off and a property to specify the number of conversions of a entity type in series needed to catch all entities, this wouldn't affect current applications and allow a fine-grained tuning. But as I said: if I understand aright ;)

I have absolutely no idea what you are talking about. I would suggest reading Christian's excellent book Java Persistence with Hibernate as it sounds like you have some real holes in your understanding of JPA/JTA etc.

And anyway, I'm talking about view layer caching - html fragment caching. But I still don't know how to make this work...

Here is a Call Tree and a Method List. I could provide the snapshot at all, but I don't know if you need it and if you use the profiler I'm using (for OS projects it's free).

You are using Seam2 right?

Pete Muir wrote on Mar 31, 2008 11:59 AM:
I don't think so. Do you mean the persistence context?

Erm. I don't know. My profiler shows, that there are about 4,500 calls for java:comp/UserTransaction. Don't know what's in there. I thought this is where all the SQL results are in.

Pete Muir wrote on Mar 31, 2008 11:59 AM:
I have absolutely no idea what you are talking about. I would suggest reading Christian's excellent book Java Persistence with Hibernate as it sounds like you have some real holes in your understanding of JPA/JTA etc.

I think reading this book is a good idea, since I'm going to tune my app.

I try to explain it another way: The EntityConverter hast to convert every single entity to a String and a String back to an entity. And to perform this conversion the EntityConverter has to lookup every single entity. I don't know, whether there's a SQL statement produced or the EntityConverter has some sort of caching. But it seems that this lookup takes a lot of time, and since I have about 1,000 entities to convert, it takes really a lot of time.

My suggestion is to convert a whole bunch of entities as soon as the EntityConverter recognizes, that it has to convert a whole DB table.

I hope it's clearer now. And as I said, it's just a suggestion based on the results my profiler shows. Don't know, if I interpret them correctly.

But to prove (or disprove ;)) that the EntityConverter has a bad performance, I'll test a page without it and do the conversion manually. I'll post my results.

Pete Muir wrote on Mar 31, 2008 11:59 AM:
And anyway, I'm talking about view layer caching - html fragment caching. But I still don't know how to make this work...

Sounds nice. I'm curious to see results once you got it.

Pete Muir wrote on Mar 31, 2008 11:59 AM:
You are using Seam2 right?

Yes, I use Seam2.

Jens Weintraut wrote on Mar 31, 2008 03:03 PM:
But to prove (or disprove ;)) that the EntityConverter has a bad performance, I'll test a page without it and do the conversion manually. I'll post my results.

OK. I just replaced

<s:selectItems value="#{helper.testcases}" var="tc" label="#{tc.ID} - #{tc.token}" />
<s:convertEntity />

by

<f:selectItems value="#{helper.testcasesSI}" />

where testcasesSI just puts all those testcases in a List<SelectItem>. The page needed 17,305ms to load with EntityConverter and now is loaded in 4,187ms. It's a real difference.

Also the calls for java:comp/UserTransaction decreased from about 4,500 to 30.

I really like EntityConverter, it's a very brilliant feature. But this results tell me it's too slow and there should be some tuning.

I try to explain it another way: The EntityConverter hast to convert every single entity to a String and a String back to an entity. And to perform this conversion the EntityConverter has to lookup every single entity. I don't know, whether there's a SQL statement produced or the EntityConverter has some sort of caching. But it seems that this lookup takes a lot of time, and since I have about 1,000 entities to convert, it takes really a lot of time.

Well, it asks JPA for each entity. Of course, if you don't have any caching set up, it has to load each entity.

My suggestion is to convert a whole bunch of entities as soon as the EntityConverter recognizes, that it has to convert a whole DB table.

How? By guesswork? I also don't really see how this will help - you have presumably queried the database for this data, and are just refering to simple, eager loaded properties. So another hit to the database shouldn't be necessary.

But to prove (or disprove ;)) that the EntityConverter has a bad performance, I'll test a page without it and do the conversion manually. I'll post my results.

Of course you will get much better performance doing it manually, thats a no brainer.

I'm not sure why you are seeing so much time in java:comp/UserTransaction - I've added this to our list of areas to investigate

I really like EntityConverter, it's a very brilliant feature. But this results tell me it's too slow and there should be some tuning.

Have a go, submit your work on JIRA in diff format :-)

Pete Muir wrote on Mar 31, 2008 05:39 PM:
Well, it asks JPA for each entity. Of course, if you don't have any caching set up, it has to load each entity.

Hmm, OK. You mentioned caching I could set up. What would you suggest?

The bean which asks the DB for those entities caches the result, but this only affects further life cycles. Is there a possibility that allows EntityConverter to benefit from the bean's cache?

Pete Muir wrote on Mar 31, 2008 05:39 PM:
How? By guesswork? I also don't really see how this will help - you have presumably queried the database for this data, and are just refering to simple, eager loaded properties. So another hit to the database shouldn't be necessary.

How? Hmm ... Beware of my ideas. I'm just guessing and my mind is a little weird from time to time :D

Isn't there a possibility for EntityConverter to ask JPA for all the entities in a table? If this could be done, it is just a small step. Introduce a parameter (set by components.xml) entityCountToLoadAllOfThem (OK OK, I'm German, it's the best name that I had ...) where the user sets just a number, say 10. When the EntityConverter.getAsString() is invoked it saves the class from the converted entity (convertedClass) and a counter countConversions. When EntityConverter.getAsString() is invoked next time and the convertedClass matches, increase countConversions. And as soon as countConversions is equal or greater than entityCountToLoadAllOfThem, just get them all and convert them.

If this would work, I think it could increase the performance, don't you think? And yeah, maybe I'm a noob to suggest such a feature, it's just an idea ;)

Pete Muir wrote on Mar 31, 2008 05:39 PM:
Of course you will get much better performance doing it manually, thats a no brainer.

Damn. And I spend my time on it :D So you recommend the use of EntityConverter only if there are not too much entities to convert?

Pete Muir wrote on Mar 31, 2008 05:39 PM:
I'm not sure why you are seeing so much time in java:comp/UserTransaction - I've added this to our list of areas to investigate

Thanks for that. Perhaps it helps to know that a lot of time is taken by NamingContext.lookup(). I don't know about the interna, so I just wanted to mention that.

Pete Muir wrote on Mar 31, 2008 05:39 PM:
Have a go, submit your work on JIRA in diff format :-)

I'll keep that in mind ;)

Jens Weintraut wrote on Mar 31, 2008 06:09 PM:

Pete Muir wrote on Mar 31, 2008 05:39 PM:
How? By guesswork? I also don't really see how this will help - you have presumably queried the database for this data, and are just refering to simple, eager loaded properties. So another hit to the database shouldn't be necessary.

How? Hmm ... Beware of my ideas. I'm just guessing and my mind is a little weird from time to time :D Isn't there a possibility for EntityConverter to ask JPA for all the entities in a table?

How? What's the concrete implementation of this? I know of no way of doing this.

Pete Muir wrote on Mar 31, 2008 05:39 PM:
Of course you will get much better performance doing it manually, thats a no brainer.

Damn. And I spend my time on it :D So you recommend the use of EntityConverter only if there are not too much entities to convert?

Well, I would always build an app for functionality first, then analyse for performance bottlenecks. At this point, I might decide to implement some custom converters, yes.

OK. Forget my first idea regarding load a bunch of entities (it's bullshit). As you said the functionality is complete, now I'm searching for bottlenecks.

And EntityConverter is such a bottleneck. But I don't want to miss it. So we had inspected its code and think we have a possible cause.

@SuppressWarnings("unchecked")
@Transactional
public String getAsString(FacesContext facesContext, UIComponent cmp, Object value) throws ConverterException
{
   init();
   if (value == null)
   {
      return null;
   }
   if (value instanceof String) 
   {
      return (String) value;
   }
   return store.put(value);
}

The problem I see here is the init() method:

private void init()
{
   if (getPersistenceContext() != null && entityManagerImpl == null)
   {
      entityManagerImpl = (EntityManager) getPersistenceContext().getValue();
      store.setPersistenceContext(entityManagerImpl);
   }
}

This method evaluates every time the value expression that provides the EntityManager. And this evaluation is very time consuming. Sending a few entities to the client doesn't matter, but in my case there are about 1,500 entities to render. And for each entity the EntityManager has to be resolved. I think this could be improved, but I don't know how, since I don't know how this stuff works.

I tried this:

private EntityManager entityManagerImpl;

private void init()
{
   if (getPersistenceContext() != null && entityManagerImpl == null)
   {
      entityManagerImpl = (EntityManager) getPersistenceContext().getValue();
      store.setPersistenceContext(entityManagerImpl);
   }
}

But it doesn't work. It results in an

java.lang.IllegalStateException: Unable to get a Persistence Context to load Entity. Make sure you have an SMPC called entityManager configured in components.xml (or have correctly configured s:convertEntity to use another SMPC).
	at org.jboss.seam.ui.converter.entityConverter.AbstractEntityLoader.get(AbstractEntityLoader.java:25)

since the SMPC can't be resolved. That's what I mean, when I say I don't know how this stuff works. Perhaps you have an idea how to avoid resolving the EntityManager for every single entity to convert.

I had slow performance with pages using entityConverter as well. However profiling tells me there is nothing wrong with this component, it is rather the combination of using rich faces component and many entities that slow down page loading. (The server takes alot of time rendering the HTML/javascript to be used on the page). I think there is no magic solution to that aside from having some sort of caching mechanism in place and/or a search/pager mechanism to avoid loading/showing too many entities on the same web page.

Besides Jens have you verified whether your entities are being lazy loaded during the render response phase? I looked for java:comp/UserTransaction and found no bottlenecks whatsoever. (all my entities are eagerly loaded and readily accessible from the persistencecontext, i'm using SMPCs.)

Regards, -Guillaume