Skip to content
This repository has been archived by the owner on Dec 11, 2020. It is now read-only.

Added batch inserts for doctrine orm populate #1781

Merged
merged 2 commits into from
Sep 2, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -409,6 +409,9 @@ Faker provides adapters for Object-Relational and Object-Document Mappers (curre

To populate entities, create a new populator class (using a generator instance as parameter), then list the class and number of all the entities that must be generated. To launch the actual data population, call the `execute()` method.

Note that some of the `populators` could require additional parameters. As example the `doctrine` populator has an option to specify
its batchSize on how often it will flush the UnitOfWork to the database.

Here is an example showing how to populate 5 `Author` and 10 `Book` objects:

```php
Expand Down Expand Up @@ -452,6 +455,8 @@ print_r($insertedPKs);
// )
```

**Note:** Due to the fact that `Faker` returns all the primary keys inserted, the memory consumption will go up drastically when you do batch inserts due to the big list of data.

In the previous example, the `Book` and `Author` models share a relationship. Since `Author` entities are populated first, Faker is smart enough to relate the populated `Book` entities to one of the populated `Author` entities.

Lastly, if you want to execute an arbitrary function on an entity before insertion, use the fourth argument of the `addEntity()` method:
Expand Down
34 changes: 31 additions & 3 deletions src/Faker/ORM/Doctrine/Populator.php
Original file line number Diff line number Diff line change
Expand Up @@ -3,27 +3,43 @@
namespace Faker\ORM\Doctrine;

use Doctrine\Common\Persistence\ObjectManager;
use Faker\Generator;

/**
* Service class for populating a database using the Doctrine ORM or ODM.
* A Populator can populate several tables using ActiveRecord classes.
*/
class Populator
{
/** @var int */
protected $batchSize;

/** @var Generator */
protected $generator;

/** @var ObjectManager|null */
protected $manager;

/** @var array */
protected $entities = array();

/** @var array */
protected $quantities = array();

/** @var array */
protected $generateId = array();

/**
* @param \Faker\Generator $generator
* Populator constructor.
* @param Generator $generator
* @param ObjectManager|null $manager
* @param int $batchSize
*/
public function __construct(\Faker\Generator $generator, ObjectManager $manager = null)
public function __construct(Generator $generator, ObjectManager $manager = null, $batchSize = 1000)
{
$this->generator = $generator;
$this->manager = $manager;
$this->batchSize = $batchSize;
}

/**
Expand Down Expand Up @@ -55,6 +71,9 @@ public function addEntity($entity, $number, $customColumnFormatters = array(), $
/**
* Populate the database using all the Entity classes previously added.
*
* Please note that large amounts of data will result in more memory usage since the the Populator will return
* all newly created primary keys after executing.
*
* @param null|EntityManager $entityManager A Doctrine connection object
*
* @return array A list of the inserted PKs
Expand All @@ -72,9 +91,18 @@ public function execute($entityManager = null)
foreach ($this->quantities as $class => $number) {
$generateId = $this->generateId[$class];
for ($i=0; $i < $number; $i++) {
$insertedEntities[$class][]= $this->entities[$class]->execute($entityManager, $insertedEntities, $generateId);
$insertedEntities[$class][]= $this->entities[$class]->execute(
$entityManager,
$insertedEntities,
$generateId
);
if (count($insertedEntities) % $this->batchSize === 0) {
$entityManager->flush();
$entityManager->clear($class);
}
}
$entityManager->flush();
$entityManager->clear($class);
}

return $insertedEntities;
Expand Down