Skip to content

Read large files, line by line, without causing memory overflow for Laravel applications.

License

Notifications You must be signed in to change notification settings

fruivita/line-reader

Repository files navigation

File Reader for Laravel applications

Latest Version on Packagist GitHub Release Date GitHub last commit (branch) GitHub Tests Action Status Test Coverage Maintainability GitHub Code Style Action Status GitHub issues GitHub repo size Packagist Total Downloads GitHub

This package, for Laravel applications, allows you to read the contents of huge files without killing your server, that is, without having to load all the contents at once in memory causing an out-of-memory errors.

The strategy used here is to read the contents of the file, line by line, optimizing the use of server resources and, most importantly, in an efficient way.

It is also possible to paginate the contents of the file, again, without having to load it entirely into memory, except the page itself.

use FruiVita\LineReader\Facades\LineReader;

$generator = LineReader::readLines($file_path);

// or

$length_aware_paginator = LineReader::readPaginatedLines($file_path, $per_page, $page);

 


Table of Contents

  1. Notes

  2. Prerequisites

  3. Installation

  4. How it works

  5. Testing and Continuous Integration

  6. Changelog

  7. Contributing

  8. Code of conduct

  9. Security Vulnerabilities

  10. Support and Updates

  11. Roadmap

  12. Credits

  13. Thanks

  14. License


Notes

⭐ Internally, this package reads the file contents using php's SplFileObject and Generators classes. In the specific case of pagination, the LimitIterator class is used to delimit the beginning and end of the content to be read.

❤️ Heavily inspired by the bcremer/LineReader package.

⬆️ Back

 

Prerequisites

  1. PHP dependencies

    PHP ^8.0

    Extensions

    composer check-platform-reqs
  2. GitHub Package Dependencies

⬆️ Back

 

Installation

  1. Install via composer:

    composer require fruivita/line-reader
  2. Optionally publish the translations

    php artisan vendor:publish --provider='FruiVita\LineReader\LineReaderServiceProvider' --tag='lang'

    The strings available for translation are as follows. Change them as needed.

    {
        "The file entered could not be read": "The file entered could not be read"
    }

    This package already has translations for en and pt-br.

⬆️ Back

 

How it works

  1. Reading a file line by line.

    use FruiVita\LineReader\Facades\LineReader;
    
    public function example()
    {
        foreach (LineReader::readLines($file_path) as $key => $line)
        {
            // $key is 0 when reading the 1st line, 1 when reading the 2nd line, and so on.
            // $line is a string with the contents of the line.
        }
    }

     

    LineReader exposes the following method to read the file line by line:

    ✏️ readLines

    use FruiVita\LineReader\Facades\LineReader;
    
    /**
     * @param string $file_path full path of the file to be read
     * 
     * @throws \FruiVita\LineReader\Exceptions\FileNotReadableException
     *
     * @return \Generator
     */
    LineReader::readLines(string $file_path);

     

    🚨 Exceptions:

    • readLines throws \FruiVita\LineReader\Exceptions\FileNotReadableException if don't have read permission on the file or it can't be found

     

  2. Reading the file by page.

    use FruiVita\LineReader\Facades\LineReader;
    
    public function example()
    {
        $per_page = 15;
        $page = 2;
    
        $length_aware_paginator = LineReader::readPaginatedLines(string $file_path, int $per_page, int $page);
        
        // The index of the items in the collection respects their position in the file using a zero-based index,
        // that is, in the example above the 1st item on page 2 will have index 15, since it is the 16th line of
        // the file and the last item on page 2 will have index 29, since it is the 30th line of the file.
    }

     

    LineReader exposes the following method to read the file by page:

    ✏️ readPaginatedLines

    use FruiVita\LineReader\Facades\LineReader;
    
    /**
     * @param string $file_path full path of the file to be read
     * @param int    $per_page
     * @param int    $page
     * @param string $page_name
     *
     * @throws \FruiVita\LineReader\Exceptions\FileNotReadableException
     * @throws \InvalidArgumentException
     * 
     * @return \Illuminate\Pagination\LengthAwarePaginator
     */
    LineReader::readPaginatedLines(string $file_path, int $per_page, int $page, string $page_name = 'page');

     

    🚨 Exceptions:

    • readPaginatedLines throws \FruiVita\LineReader\Exceptions\FileNotReadableException if don't have read permission on the file or it can't be found
    • readPaginatedLines throws \InvalidArgumentException if per_page or page is less than 1

⬆️ Back

 

Testing and Continuous Integration

composer analyse
composer test
composer coverage

⬆️ Back

 

Changelog

Please see CHANGELOG for more information on what has changed in each version.

⬆️ Back

 

Contributing

Please see CONTRIBUTING for more details on how to contribute.

⬆️ Back

 

Code of conduct

To ensure that everyone is welcome to contribute to this open-source project, please read and follow the Code of Conduct.

⬆️ Back

 

Security Vulnerabilities

Please see security policy how to report security vulnerabilities or flaws.

⬆️ Back

 

Support and Updates

The latest version will receive support and updates whenever the need arises. The others will receive updates for 06 months after being replaced by a new version and then discontinued.

Version PHP Release End of Life
1.0 ^8.0 09-05-2022 dd-mm-yyyy

🐛 Found a bug?!?! Open an issue.

⬆️ Back

 

Roadmap

✨ Any new ideas?!?! Start a discussion.

The following list contains identified and approved improvement needs that will be implemented in the first window of opportunity.

  • n/a

⬆️ Back

 

Credits

⬆️ Back

 

Thanks

👋 Thanks to the people and organizations below for donating their time to build the open-source projects that were used in this package.

💸 Some of these people or organizations have some products/services that can be purchased. If you can help them by buying one of them or becoming a sponsor, even for a short period, you will help the entire open-source community to continue developing solutions for everyone.

⬆️ Back

 

License

The MIT License (MIT). Please see the License File for more information.

⬆️ Back

About

Read large files, line by line, without causing memory overflow for Laravel applications.

Topics

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Packages

No packages published

Languages