Make path to source files relative

[robwierzbowski]

Tested with many combinations in browser. 👌.

Doesn't normalize Windows paths yet. Putting up for review but want to add that before merge.

[robwierzbowski]

@UltCombo Want to test this in Win? I think it should work, but it will have Windows style paths.

[robwierzbowski]

...aaaaand now it should have the correct browser-style paths on Windows too.

[UltCombo]

@robwierzbowski No problem, I'm testing right now. =]

The path separators are correct, but the relative paths seem to be going a couple directories above than it should. I'll prepare some test cases and read the source to see if I can figure out the issue.

[robwierzbowski]

If you give me a source glob, dest, and file structure I'll test on my side.

[UltCombo]

@robwierzbowski alright, you can clone https://github.com/UltCombo/gulp-ruby-sass-maps-test

[robwierzbowski]

Ah, it's a __dirname issue. Thanks.

[robwierzbowski]

Ok, @UltCombo, give that a spin. Tested on an external project, should be working well.

[robwierzbowski]

I think next on the list of things to do is update our tests to better mirror a real usage situation. :P.

[UltCombo]

@robwierzbowski looks better, but seems like there is still an extra ../ in the sources there -- using the same test repo above with the be8ee9960d18e4cef6c730910dd5c35ef606d2cf commit.

[robwierzbowski]

I served it up in a little application, seems to be working for me.

[robwierzbowski]

Forked your project, added an index for testing: https://github.com/robwierzbowski/gulp-ruby-sass-maps-test/tree/7c419e7a8f2ace6741464f16d79d67c48cec894e

[UltCombo]

@robwierzbowski By any chance, did you serve the repo root in the root of your web server? If so, the extra ../ is dropped when the browser resolves the URL. Try serving the application in a sub directory e.g. http://localhost/gulp-ruby-sass-maps-test/ and the map will resolve to http://localhost/scss/app.scss.

[UltCombo]

Picturing @robwierzbowski at the moment:

No, but seriously, I can take a deeper look at this in case you're too tired/fed up with the sourcemapping stuff. :P

[robwierzbowski]

Haha. Well there are some complications that I didn't realize. Working through them now. I'd like to get this and the other issue wrapped up and then release a new version ASAP, then just let it coast for a while.

[UltCombo]

Oh I see, v0.6.0 looks really promising and the source maps are almost functional. Very nice job so far.

[GC-Mark]

Good work on fixing this source map issue...looking forward to it being merged

[UltCombo]

Consider using uri-path instead of slash in order to properly encode the relative URL paths. See these discussions: gruntjs/grunt-contrib-uglify#222 (comment) and gruntjs/grunt-contrib-uglify#175 (comment)

[robwierzbowski]

Will do, thanks.

[robwierzbowski]

After working on this with a couple more possible setups, I think the best solution is just to tell the user to supply the relative path to their source files. Revisions, examples, and reasoning to come.

[UltCombo]

@robwierzbowski I'm okay with that, your suggestion also solves use cases where the webserver URLs do not correspond to the file system's.

The user can do uriPath(path.resolve()) themselves to keep the code DRY when webserver URLs correspond to the underlying file system (majority of use cases).

[UltCombo]

@robwierzbowski if I understand correctly, then your current suggestion would be supplying the sources array ourselves, e.g.:

var cssSrc = 'scss',
    cssDest = 'public/css',
    globbingPattern = cssSrc + '/*.scss';

var sources = glob.sync(globbingPattern).map(function(srcPath) {
    return uriPath(path.relative(cssDest, srcPath));
});

gulp.src(globbingPattern)
    .pipe(sass({
        sourcemap: true,
        sources: sources
    }))
    .pipe(gulp.dest(cssDest));

Though, now this seems very fragile. If the sources array is not in the correct order, the tokens will mapped to the wrong files.

Maybe a callback function to treat the sources array could be more bullet proof.

Though, I wonder what are complications about this relative path resolution. Would you mind if I try to help fix/workaround this issue?

[robwierzbowski]

That's not really my suggestion, and I don't think it will be a problem. I'm working this week, give me a day or so to get my code up and we can go from there.

[UltCombo]

Alright then, guess I'll check how is Broccoli's source mapping going for the time being.

[robwierzbowski]

We have two situations, the combination of which prevent us from doing any special magic to find out the relative path between output CSS and sourcemaps.

1. Output directory can be anywhere in the project folder, relative to source directory
2. Server webroot can be anywhere.

The only solution to the sourcemap path issue I see is having the user specify the relative path from their output directory as seen by the server to their source directory as seen by the server. We can join this with some knowns to get the correct relative path.

Caveats: the source files must always be visible to the web browser, under the webroot. If the files are outside of the webroot the user will need to use some connect middleware to expose them. In a Yeoman type project this would actually be easy: .tmp/styles has the output CSS, app/styles has the SCSS, and connect serves .tmp and app on top of each other, so the relative path between .tmp/styles and app/styles is .. sass({ sourcemaps: true, sourcemapPath: '.' }).

I need to expand the docs a little and do some more tests, but I think this is where the feature will land. It's not super easy to explain but I'm pretty sure there's no magic we can do behind the scenes without knowing both the output dir and the webroot.

[robwierzbowski]

@UltCombo You were right, I found where the extra ../ was being set. All of that logic is up to the user now though, so no need to worry about it. uri-path looks good to use too, thanks.

[UltCombo]

The only solution to the sourcemap path issue I see is having the user specify the relative path from their output directory as seen by the server to their source directory as seen by the server.

Yep, that makes perfect sense.

Caveats: the source files must always be visible to the web browser, under the webroot.

That doesn't seem to be a caveat, rather it is by design -- logically, it is impossible for the browser to succeed in requesting a file which the web server is not serving.

In a Yeoman type project [...]

Oh, not only Yeoman, even with just Express it is not too uncommon to serve aliased static resources, e.g. app.use('/sources', express.static(__dirname + '/scss'));, so the file system mapping is not always equal to the web server routes. This is easily solved with your proposed sourcemapPath though. ;)

You were right, I found where the extra ../ was being set. All of that logic is up to the user now though, so no need to worry about it. uri-path looks good to use too, thanks.

No problem. =] Though, if the user is inputting the sourcemapPath him/herself, I wonder if s/he shouldn't/wouldn't input an already properly encoded URL path. E.g. when you copy from the address bar, the URL will be properly encoded already. Double encoding it would be an issue. Well, I guess this is just a matter of preference and documenting, shouldn't give too much thought about rare corner cases.

[UltCombo]

Or, I could add a doubleEncode: false option to uri-path... Though, %'s are actually valid in Windows filenames, gah.

[robwierzbowski]

Hm, double encoding wouldn't be great. I'm using path.join so switching to uri-style slashes is still relevant.

[robwierzbowski]

If you were going to add logic to uri-path, I don't think it would need to be an option. No one wants to double encode, right?

[UltCombo]

Oh I see. It would be hard, if not impossible, to do proper double-encoding prevention as % signs are valid in file paths (e.g. it is valid to have %20 as part of a directory/file name in Windows).

My initials thoughts were to decode any % escape sequences and then encode the URL path -- this way, no double encoding occurs and it handles cases where only part of the URL path are properly encoded. But this would break when the file path/name actually contains the % sign. That may be a rare use case, but you know, being less than 100% robust doesn't feel good.

[UltCombo]

The two options that would work 100% for gulp-ruby-sass which I can think of:

• Expect a properly encoded relative URL path (that would have all special characters %-encoded and forward slashes as path fragment separators) and then gulp-ruby-sass just has to run uriPath on the filename before concatenating it to the path. E.g. sourcemapPath + uriPath(fileName). Though, path.join would still be nice for sourcemapPath to accept paths with and without the trailing slash (a backslash->forward slash replacement would have to follow after path.join for Windows then though).
• Or, expect a decoded URL path and keep encoding everything as your latest commit is already doing.

[robwierzbowski]

OK, just some better docs and this should be good to go.

[UltCombo]

Your logic seems pretty robust, though I'm not sure what the "original sourcemap path" actually means. I'll pull your commit and run some tests as well. =]

[UltCombo]

Oh, by "original sourcemap path" you mean the sourceMappingURL I assume. Sorry, I'm a bit slow today. And yes, the sourceMappingURL is perfectly encoded, however there's one part that is still missing the proper encoding:

1. ....
2. a relative path to the base of the CSS directory, which is only ever ../s — no encoding needed
3. the user provided relative path, which we can ask to be encoded
4. The sources' filenames

But I guess including uri-path just for that is unnecessary, you can just run the filename through encodeURIComponent.

//cc @robwierzbowski

[robwierzbowski]

The sourceMappingURL contains the filenames and is written by Sass, so any explosions that happen there can be handled upstream. I think it's 👌

I'm happy with how this is working. I really appreciate the fast testing and review, BTW.

[UltCombo]

You're welcome. But oh, guess I wasn't very clear. I meant that the .map file's sources array's filenames are not properly encoded. E.g. assume we're compiling app foo.scss into app foo.css, the sources array will contain 'app foo.scss', where it would expect 'app%20foo.scss'.

The sourceMappingURL handles this correctly as it is done by Sass, but gulp-ruby-sass's sources rewriting is not encoding the filenames correctly.

[UltCombo]

Gawd, I'm having a hard time to articulate statements together, I should probably sleep a bit more at night. Anyway, edited my previous comment for clarity.

[robwierzbowski]

Running sass directly on a file called fixture a.scss, the sourcemap sources output is ["../../fixture/fixture a.scss"]. As long as this is existing behavior, I think it's good to leave. If it causes issues we can file an issue with Sass.

[UltCombo]

@robwierzbowski the source map spec expects properly encoded URL(s) in the sources array. Spaces cannot appear unencoded in a URL.

I know this is a rare use case, but we're so close to be fully sourcemap spec compliant. =]

[robwierzbowski]

If it's not compliant it needs to be fixed upstream so the fix bubbles down to everybody. You seem to know a ton about the sourcemap spec — want to file an issue?

[UltCombo]

@robwierzbowski are you sure this is an upstream bug? I thought this was a bug with this plugin's source map rewriting, I'll double check it with vanilla Sass.

[UltCombo]

@robwierzbowski yep, it is actually an upstream bug. My bad.

[robwierzbowski]

Running sass directly on a file called fixture a.scss, the sourcemap sources output is ["../../fixture/fixture a.scss"]

That was on Mac with Sass 3.3.8. Might be different on Win, but I'm guessing not.

[UltCombo]

@robwierzbowski ohh, I somehow skipped the "Running sass directly on a file" part, my bad. Haven't slept very well lately, guess I'll file an issue in the Sass repository after I take a nap. Your PR looks great, good job. =]

[robwierzbowski]

No worries!

[robwierzbowski]

OK @sindresorhus, this is ready to go.

[sindresorhus]

@robwierzbowski awesome :)

[danxshap]

Hey guys, very new to Sass sourcemaps and crazy coincidence that this issue is being worked on right when I realize I'm having this problem! (I think).

I'm running a Django app locally and my Sass partial imports look like this:
@import '../../../../accounts/static/accounts/scss/profile';

That works fine & well for grabbing the partial and generating the compiled CSS, but my local webserver has this file available at 127.0.0.1:8000/static/accounts/scss/_profile.scss (this is how static directories in Django are supposed to work), and the sourcemap tells Chrome/DevTools that the file is at 127.0.0.1:8000/accounts/static/accounts/scss/_profile.scss which is a 404 so I can't view/edit the source in DevTools.

1. Am I correct in understanding that this is this the same issue you guys are discussing here?
2. Is this an issue with how Sass itself works? Or an issue with how my dev environment/tools are working?
3. I've never heard of gulp before, but at first glance it seems like a replacement for https://github.com/cyberdelia/django-pipeline which is what I'm using to concatenate/minify my static files for deployment -- if it's not a Sass issue and indeed is a special feature you guys are adding via gulp, does that mean I'm SOL if I'm not using gulp?
4. Thank you very much for any advice here!

[robwierzbowski]

Gulp is a separate build system, not connected to django in any way. The
issue is almost certainly with the django server setup and not sass
itself. I'd check on the django issue tracker and message boards -- I don't
have any experience with it. Good luck!

On Saturday, July 5, 2014, Daniel Shapiro notifications@github.com wrote:

Hey guys, very new to Sass sourcemaps and crazy coincidence that this
issue is being worked on right when I realize I'm having this problem! (I
think).

I'm running a Django app locally and my Sass partial imports look like
this:
@import '../../../../accounts/static/accounts/scss/profile';

That works fine & well for grabbing the partial and generating the
compiled CSS, but my local webserver has this file available at
127.0.0.1:8000/static/accounts/scss/_profile.scss (this is how static
directories in Django are supposed to work), and the sourcemap tells
Chrome/DevTools that the file is at
127.0.0.1:8000/accounts/static/accounts/scss/_profile.scss which is a 404
so I can't view/edit the source in DevTools.

1. Am I correct in understanding that this is this the same issue you
   guys are discussing here?
2. Is this an issue with how Sass itself works? Or an issue with how
   my dev environment/tools are working?
3. I've never heard of gulp before, but at first glance it seems like
   a replacement for https://github.com/cyberdelia/django-pipeline which
   is what I'm using to concatenate/minify my static files for deployment --
   if it's not a Sass issue and indeed is a special feature you guys are
   adding via gulp, does that mean I'm SOL if I'm not using gulp?
4. Thank you very much for any advice here!

—
Reply to this email directly or view it on GitHub
#68 (comment)
.

Rob Wierzbowski
@robwierzbowski http://twitter.com/#!/robwierzbowski
http://github.com/robwierzbowski
http://robwierzbowski.com

[UltCombo]

@danxshap

First off, source maps are in a state of chaos, to be honest. Each tool and each task runner/build tool/asset packager handles source maps differently.

1. Am I correct in understanding that this is this the same issue you guys are discussing here?

Yeah, this plugin has just added a "non-standard" sourcemapPath setting in order to solve use cases such as yours.

2. Is this an issue with how Sass itself works? Or an issue with how my dev environment/tools are working?

See the Sass Documentation:

:sourcemap

When set to true, causes Sass to generate standard JSON source maps alongside its compiled CSS files. These source maps tell the browser how to find the Sass styles that caused each CSS style to be generated. Sass assumes that the source stylesheets will be made available on whatever server you’re using, and that their relative location will be the same as it is on the local filesystem. If this isn’t the case, you’ll need to make a custom class that extends Sass::Importers::Base or Sass::Importers::Filesystem and overrides #public_url.

As you can see, these use cases are often easier to workaround in the build tool/asset packager.

3.I've never heard of gulp before, but at first glance it seems like a replacement for https://github.com/cyberdelia/django-pipeline which is what I'm using to concatenate/minify my static files for deployment -- if it's not a Sass issue and indeed is a special feature you guys are adding via gulp, does that mean I'm SOL if I'm not using gulp?

I don't have experience with django-pipeline, sorry. If it is pluggable, that is, if it accepts adding 3rd-party plugins/tasks to the packaging logic, you could (in gulp terminology) pipe the maps into a task to rewrite their sources array (.map files are just JSON) before outputting them to disk.

It is worth opening an issue in their repository, in my opinion.

In the worst case, you could just watch the .map files and rewrite them using Gulp. =]

[danxshap]

@UltCombo awesome thanks for that information! Just to reiterate/confirm my understanding: this plugin here is opening up the sourcemap file that Sass generates and it is changing the paths in the sources JSON to resolve this issue, right? i.e. that is what django-pipeline would have to do to emulate the functionality you guys are adding.

[UltCombo]

@danxshap yes, that's the logic which @robwierzbowski has implemented to fix this issue. You can see it here:

gulp-ruby-sass/index.js

Lines 12 to 43 in 0a2359c

	function rewriteSourcemapPaths (cssDir, relPath, cb) {
	var glob = require('glob');
	glob(path.join(cssDir, '**/*.map'), function (err, files) {
	if (err) {
	cb(err);
	return;
	}
	eachAsync(files, function (file, i, next) {
	fs.readFile(file, function (err, data) {
	if (err) {
	next(err);
	return;
	}
	var sourceMap = JSON.parse(data);
	var stepUp = path.relative(path.dirname(file), cssDir);
	// rewrite sourcemaps to point to the original source files
	sourceMap.sources = sourceMap.sources.map(function (source) {
	var sourceBase = source.replace(/\.\.\//g, '');
	// normalize to browser style paths if we're on windows
	return slash(path.join(stepUp, relPath, sourceBase));
	});
	fs.writeFile(file, JSON.stringify(sourceMap), next);
	});
	}, cb);
	});
	}

[danxshap]

Great, really appreciate the info. Thank you and have a great rest of your weekend!

[UltCombo]

No problem, you too. =]

[OliverJAsh]

We need to talk about source maps.