We have moved our forum to GitHub Discussions. For questions about Phalcon v3/v4/v5 you can visit here and for Phalcon v6 here.
  1. Home
  2. ORM

Memory accumulation of Model::_groupResult

Hi All

I have an application where large amounf of data is loaded into the database by CLI tasks that runs frequently. The case is that we loop over the list and insert the data into the database. The model we use to create the data in the Db performs a number of sum calculations, from other models. From time to time this job runs out of memory, so we have been running a Xdebug profiling, which is seen in the dump below.

It seems that Model::_groupResult accumulates quite alot of memory with an increasing number of calls, the main callers from this is the Model::sum function, is there anyway to reduce/prevent this or release memory?

I'm not quite sure of the functionality of _groupResult, but would it be better to write a PHQL statement wich include the sum function or would this be same?

Profile

Thanks in advance



10.4k
edited Oct '20

I've tried to do some database imports from one DB to Another, but I've fallen into similar problem: getting a lot of Model instances from DB even if I later null them / unset, consumes a lot of memory.

I've not found any method to free memory in Phalcon.

That does not free memory:

$model = \MyModel::findFirst(query);
$model = null;

or

unset($model);

Finally, I've deceided to realize that import procedure in pure PDO, maybe you should do the same, Phalcon ORM is probably not build to operate on large volumes of data.



125.7k

For operations like iterating through a resultset, Phalcon just loads one row at a time - so it's very memory smart. However, for doing something like a SUM operation, it needs to load all the rows before it can operate - that takes lots of memory.

SUM functionality will work much better if you get the database to do it. I'd use PDO & raw database queries instead of the ORM.