Automatic timezone conversion in Eloquent?

[redelschaap]

I am developing an application that has an API where consumers can send requests containing dates. They would post these dates in ISO format, containing a timezone reference. Currently, we have to do a conversion ourselves to the application's timezone (typically UTC). The framework does not convert the timezone when storing dates into the database. To be honest, I would expect this kind of magic from Laravel.

Let's imagine a model with a date cast:

class BlogPost extends Model {
    protected $casts = [
        'published_at' => 'datetime',
    ];
}

When inserting a model from an API request that accepts an ISO formatted datetime string, we need to store this as follows:

BlogPost::create([
    // ...
    'published_at' => Carbon::parse($request['publishedAt'])->setTimezone(config('app.timezone'));
    // ...
]);

It would be very nice, and would feel much more like Laravel, when we can leave out the setTimezone() part.

Now, I know that this has already been discussed before and until now the message was that this behavior was intended. But with all the good magic Laravel does on other parts of the framework, it really feels weird that It lacks this kind of magic. At least for me, and given the number of questions about this, also for many other developers.

Would the framework be open for automatic timezone conversion? Perhaps in the next major release, since this will probably break some applications. Although it should only break things when developers currently adjust time or anticipate on this behavior in another way than setting the timezone themselves. I can also think of an option where this would be an opt-in or opt-out function, just like with the preventLazyLoading or preventSilentlyDiscardingAttributes features.

I am willing to write a PR, but only when it has a chance of getting accepted of course. Please let me know!

[shaedrich]

When you use form requests, you can do

class MyFormRequest
{
    public function rules(): array
    {
        return [
            // …
        ];
    }

    public function passedValidation()
    {
        $this->merge([
            'publishedAt' => $this->date('publishedAt')->setTimezone(config('app.timezone')),
        ]);
    }
}

I know, it's not a solution, but it can be a workaround, especially if you use this in a base class.

[Sophist-UK]

@driesvints Dries

I think we all understand the need for backwards compatibility so anything like this would need to be fully backwards compatible. I note on further research that some databases support blueprint dateTimeTz and timestampTz fields, so database support is probably already covered. Hence my suggestion below to provide standard casts for this (that would have to be explicitly added to a model) and which would therefore be fully backwards compatible for existing apps. But I can see no reason why this cannot be added using a package.

[redelschaap]

@driesvints Well, I'm not really talking about cases where the app.timezone has changed. I agree that the timezone should stay on UTC. When it comes to storing dates in the database, the current behavior is actually confusing to me as a developer.

Let me try to demonstrate for a final attempt. Let's say I retrieved a datetime value in another timezone and I want to store this date on a model:

$date = Carbon::parse('2024-12-04T12:34:56+02:00');

$blogPost = BlogPost::create([
  'title' => 'Lorem ipsum dolor sit amet',
  'published_at' => $date
]);

Dumping some variables will give me the following result:

BlogPost {
  id: 1
  title: "Lorem ipsum dolor sit amet"
  published_at: "2024-12-04 12:34:56"
  created_at: "2024-12-04 20:26:35"
  updated_at: "2024-12-04 20:26:35"
}

$date->toIso8601String() => "2024-12-04T12:34:56+02:00"
$date->setTimezone('UTC')->toIso8601String() => "2024-12-04T10:34:56+00:00"
$blogPost->published_at->toIso8601String() => "2024-12-04T12:34:56+00:00"

Storing the datetime value in published_at like this will loose the timezone information. Since the app timezone is set to UTC, I would expect datetimes to be converted to UTC. However, the next time I retrieve the model, the timezone is just discarded and the time is 2 hours off.

---

I understand that a change like this would break tons of applications. That's why I suggested an opt-in like the preventSilentlyDiscardingAttributes feature. But let's be honest, with all the magic Laravel brings, we can at least expect timezone conversion when storing datetime values from different timezones, right?

[driesvints]

@redelschaap You need to do:

$blogPost = BlogPost::create([
  'title' => 'Lorem ipsum dolor sit amet',
  'published_at' => $date->toUtc(),
]);

And if you want to remember the timezone it originally was you need to do:

$blogPost = BlogPost::create([
  'title' => 'Lorem ipsum dolor sit amet',
  'published_at' => $date->toUtc(),
  'published_at_timezone' => $date->timezone,
]);

Then re-apply it on retrieval if you want it to be in the same timezone again.

Since the app timezone is set to UTC, I would expect datetimes to be converted to UTC.

That's simply not how things work. It's intended not to.

But let's be honest, with all the magic Laravel brings, we can at least expect timezone conversion when storing datetime values from different timezones, right?

No, we cannot 😅 this is just a wrong assumption sorry. There are tons of apps out there who simply don't take the timezone of a datetime object into account and just work with the date and time of the values without expecting it to convert at all.

app.timezone isn't and never was intended to convert time stored on Eloquent models. In fact it has nothing to do with Eloquent at all. The docs nowhere claim it either. This setting is purely for new datetime objects being created in your app and for things like queues, the task scheduler and console commands.

You are responsible yourself for making sure you store datetime objects in the timezone you expect.

[Sophist-UK]

Whilst I agree with @driesvints Dries here about responsibility for handling timezones correctly, the entire purpose of frameworks like Laravel and their add-on package ecosystems is that they provide standard functionality to avoid web developers from having to reinvent the same wheel time and time again.

Whilst it is absolutely the responsibility of each web developer to handle timezones according to their own needs - and sites can have very different needs preventing any sort of default handling - that doesn't mean that Laravel (and the wider package ecosystem) shouldn't provide some standard functionality and documentation to make it much easier for web developers to do this right (and see my comments below).

I don't think that it matters that much whether this functionality and documentation is delivered using a package or by Laravel itself - and Taylor's policy is for this type of thing to be delivered using packages unless it becomes such common usage that it should be rolled into Laravel itself.

So, if there is consensus that such a package is needed, let's stop lobbying the Laravel team to include in Laravel and instead let's find someone willing to write and maintain a suitable package that provides the casts and any other common functions together with documentation to save people reinventing the wheel here.

As already offered below, if I ever get the time to write my open source Laravel project then I will need this functionality myself, and if a suitable package doesn't exist at that point I will write something and spin it off as a separate and independent open source package. But if anyone else wants to create such a package in the meantime, they should go for it.

[redelschaap]

Whilst I agree that it is the developer's responsibility to make sure dates are stored in the correct timezone, I still think the framework could offer a solution to make this easier.

I think this is very similar to the missing attributes. It is the developer's responsibility to not use any attributes that don't exist. However, the framework offers a way to make this easier to prevent.

But I'll accept that the maintainer have a different opinion about this. If I have the time, I am also open to make this as a package.

[Sophist-UK]

In a Joomla plugin I used to use a decade or so ago, timezone handling was one of the biggest cause of issues and the biggest headache for developers to get right.

In my opinion:

• If Carbon doesn't already have date-time conversions between time zones (handling daylight savings times based on either the date & time if the date is provided or a date provided separately or today's date if not date provided), then this should be added.
• Laravel should have functionality as standard for casting date-times to Zulu to store in a database and (obviously) in reverse when retrieving from the database plus built-in functionality to hold and store a time-zone alongside a database date-time field plus automatic conversion to/from this stored timezone or to a user's profile / session time-zone where a timezone is not stored alongside.

I haven't got the time to check whether these functions are already provided in Carbon or Laravel.

[Sophist-UK]

From a Laravel perspective this can almost certainly be provided by a package - and at some point I might even get around to writing it.

[redelschaap]

Actually, I tink dates should just be stored in the timezone configured in app.timezone, which is set to UTC for most applications. This keeps the values simple (in one format) and this does not require any other level of database support.

My point is that the framework could easily do the ->setTimezone('UTC') calls that we now have to do manually. Carbon already has timezone conversion (handling daylight savings times) out of the box, so that part is covered.

[Sophist-UK]

See @driesvints Dries answer above as to why app.timezone is intended for a different usage.

My point is that the framework could easily do the ->setTimezone('UTC') calls that we now have to do manually.

This would be completely backwardly incompatible with all existing databases that haven't done this manually already. So it is never going to happen.

We just need to make timezone conversions much easier - by which I mean to find a way to make single explicit declaration in the model that:

1. If the timezone is not explicitly stored in the database alongside the datetimestamp, define the timezone that is used for the data stored in the database.

2. Define the Model's field's timezone to be used for automatic conversion back and forth either using a web site default (which could be app.timezone() or similar), or e.g. a session variable that holds the user specific timezone.

3. Undertaking the automatic conversion between database and model. This would normally be done in Laravel using casts.

4. Store the local timezone that the model now uses as a separate attribute in the model (separately from the attribute for any timezone stored in the database). This might best be done by sub-classing Carbon classes to hold this information and to provide methods to convert between timezones.

5. Handles all the Blueprint date-time field types extremely simply, equally well and in a consistent manner:

   • dateTimeTz
   • dateTime
   • date
   • timeTz
   • time
   • timestampTz
   • timestamp
   • timestampsTz
   • timestamps

6. Handles all the variant use cases including backwards compatibility for data already stored (i.e. stored in app.timezone values), difficulties of meeting times used by people in different timezone possibly one with and one without DST (since DST changes in spring & autumn occur on different dates in different countries), handles DST or not as requested including both current algorithmic knowledge of DST change dates and a historical database of DST change dates where they were historically different) etc. etc. etc. (IMO THIS IS NON TRIVIAL!!!!)

7. Existing code would need to continue to work as it does now - and if a developer wants to use this comprehensive timezone functionality then they will need to explicitly code for it. The objective is to make this explicit usage as simple and automated as possible.