Skip to content

Don't use this yet. Modern character encoding detection and conversion using Intl and ICU.

License

Notifications You must be signed in to change notification settings

simplepie/text-encoder

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

36 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation



TextEncoder

An implementation of the WHATWG Encoding Specification in PHP.

The UTF-8 encoding is the most appropriate encoding for interchange of Unicode, the universal coded character set.

We agree with this statement. The goal of this project is to detect the current encoding of a string or document. If it isn't UTF-8, make it easy to convert to UTF-8.

Badges

Health

Open Issues Pull Requests Contributors Repo Size GitHub Commit Activity GitHub Last Commit

Quality

Travis branch Coveralls Code Quality Symfony Insight

Social

Author Follow Blog Stars

Compliance

License

Workflow

Before we can convert content to UTF-8, we first need to understand the current character encoding so that we know what we need to convert from. In order to determine this, we rely on a few specific signals:

  1. The Content-Type header in an HTTP response.
  2. The XML prologue.
  3. Well-known byte sequences in certain encodings.

Since character encodings can be fairly esoteric, and a understanding of encodings is limited, it is common that an HTTP response or XML prologue will lie about their encoding. As such, while they may be helpful, content sniffing provides the most confident results.

Once we have high confidence in the input encoding, converting to UTF-8 becomes trivial.

Specifications

Coding Standards

PSR-1/2/5/12/19 are a solid foundation, but are not an entire coding style by themselves. We automate a large part of our style requirements using PHP CS Fixer and PHP CodeSniffer. (The things that we cannot yet automate are documented in the SimplePie NG Coding Standards.)

These can be applied/fixed automatically by running the (lightweight) linter:

make lint

Additionally, in our quest to write excellent code, we use a variety of tools to help us catch issues with what we've written, including:

Type Description
Linting Tools PHP CS Fixer, PHP CodeSniffer
QA Tools PDepend, PHPLOC, PHP Copy/Paste Detector, PHP Code Analyzer
Static Analysis Phan, PHPStan, Psalm, PHP Dependency Analysis

These reports can be generated by running the (heavyweight) analyzer:

make analyze

Please Support or Sponsor Development

The SimplePie project is a labor of love. Development of the next-generation of SimplePie was started in June 2017 as because it's a project I love, and I believe our community would benefit from this tool.

If you use SimplePie — especially to make money — it would be swell if you could kick down a few bucks. As the project grows, and we start leveraging more services and architecture, it would be great if it didn't all need to come out of my pocket.

You can also sponsor the development of a particular feature. If there's a feature that you want to see implemented, and I believe it's the right fit for the SimplePie project, you can sponsor the development of the feature to get it prioritized.

Your contributions are greatly and sincerely appreciated. See the Sponsor button along the top of the page for more information.