Idea: We should do a branch prior to release (we do this anyway) and then commit to that branch a minimization of all JS and CSS, so trunk always has pretty-formatted, and release branches always have minimized.
Or this might not be needed with Andrew’s new stuff.
There are several updates to the script loader currently in WordPress 2.8-bleeding-edge that enhance and optimize loading of external JavaScript and CSS files.
Probably the most important change is that scripts can be queued for loading in the footer for both the admin and the front-end. This is done with an optional argument. To enqueue a script for the footer:
wp_enqueue_script( 'name', 'url/to/file.js', array('dependency'), 'version', true );
where “true” means enqueue for the footer (“false” is the default and is optional).
When a script is enqueued for the footer all dependencies will be added (if not already present) and will be printed before the script. Some may be in the head, others also in the footer. By default only jQuery is printed in the head but when a script is enqueued for the head, all dependencies would also be printed in the head. Almost all external scripts would run onload or after the page has loaded, so there’s no real need to queue anything for the head.
Scripts queued for the front-end footer depend on wp_footer(); being present in the current theme. Unfortunately some themes don’t include it. The best way to remedy this would be to bring awareness among users and theme designers as suggested by several plugin developers.
To make queueing of scripts easier two new actions have been added: "wp_enqueue_scripts" that runs in the front-end head where all is_page(), is_home(), etc. functions are available and "admin_enqueue_scripts" that runs in the admin head and has the current page hook as argument, so scripts can be queued only for specific pages.
Another major new feature is that all core admin scripts are concatenated and compressed before sending them to the browser. This feature can easily be extended to include scripts added by plugins and to use server side caching, however that would require some changes to the server settings (.htaccess on Apache).
Since compression from php can be problematic on some hosts there are several “switches” (constants) that manage it: define('CONCATENATE_SCRIPTS', false); would turn off both concatenating and compressing of all scripts. It’s intended for script debugging, define('COMPRESS_SCRIPTS', false); can be used to turn off compression for JavaScript and define('COMPRESS_CSS', false); for CSS files. Compression is set to “deflate” by default since it’s faster and uses a little less server resources. Gzip can be forced by setting define('ENFORCE_GZIP', true);
There is a test if compressing from php works as expected on the server and whether the server compresses scripts by default. It runs only once and saves the result in an option “can_compress_scripts”. It would run again if the option is deleted.
In addition all core scripts are minified. All custom scripts are included in two versions: .dev.js is the non-minified script and .js is the minified one. The constant define('SCRIPT_DEBUG', true); would load the .dev.js versions of the scripts making them easier to debug.
Possible changes: removing the COMPRESS_CSS switch and using only COMPRESS_SCRIPTS, using deflate for compression but adding the gzip file header and serving it as “Content-Encoding gzip” since it seems more compatible with the various web servers and proxyes (all modern browsers support deflate well).
Hi Andrew,
I also blogged about this 2 weeks ago, http://lesterchan.net/wordpress/2009/01/26/loading-javascript-in-footer-in-wordpress-28/ =D
I was hoping for concatenating API so that plugin author can use it. But I saw the path of script-loader.php, it seems to be in the /wp-admin/ folder. If it is extended to plugins, I think it would be good to move to /wp-includes/
In wp-includes/script-loader.php, the
line $scripts->add( ‘jquery’, ‘/wp-includes/js/jquery/jquery.js’, false, ‘1.2.6-1′)
accounts for about 2 seconds of total page load time on any simple blog such as mary.wordpress.com. With it, the load time is: 4.94s; without it 2.88
Specifically,
http://s2.wordpress.com/wp-includes/js/jquery/jquery.js?ver=1.2.6-1 153ms
http://mary.wordpress.com/wp-content/plugins/mshots/jquery-ui-personalized-1.6rc2.packed.js?ver=MU 335ms
http://mary.wordpress.com/wp-content/plugins/mshots/jquery.cluetip.min.js?ver=MU 204ms
http://mary.wordpress.com/wp-content/plugins/mshots/mshots.js?ver=4 434 ms
A principle of frontend is to load js at the end of page, or as further down as possible. Not sure how further down we can move it, just throw in some thoughts here.
At the moment I am having issues with the jquery being loaded in the scriptloader. I include the 1.2.6 version via Google (http://ajax.googleapis.com/ajax/libs/jquery/1.2.6/jquery.min.js), and a dependent script will only work (frontend) if I take out the jquery from the scriptloader. This however will make the write post in the dashoard useless.
So I would like to be able to conditionally include jquery in scriptloader for admin and/or frontend.
Does this make sense?
I had a problem with my theme after updating to 2.8, adding true solved it, thanks a lot 
Good post, but specifically, how would I go about compressing and concatenating scripts on the front end ?
Thanks
The “first run” of the script loading optimization is in trunk. It uses both methods: splits the scripts queue in head and footer parts, then concatenates and compresses them before sending them to the browser. Most CSS is also concatenated and compressed.
There are two new constants that disable concatenating and compression: CONCATENATE_SCRIPTS and COMPRESS_SCRIPTS. Setting the first to false would disable both concatenating and compression, the second disables compression only. There is also a simple AJAX method to test if compressing from PHP works as expected on the server. It is run only once and the result is saved as an option. It will run again if that option is deleted.
For plugin authors there are two new actions in the admin footer: do_action('admin_print_footer_scripts'); and do_action("admin_footer-$hook_suffix");. The order of execution is:
"admin_footer" can be used to print scripts before the default footer scripts"admin_print_footer_scripts" used to print the default scripts followed by any external scripts that were queued for the footer"admin_footer-$hook_suffix" can be used to print scripts that should appear on a specific admin page onlyThe preferable way for plugins to add scripts to admin pages would be either to enqueue them properly (which is a must if the script depends on a default script) or to use the $hook_suffix hooks to add them only where needed.
New hook(s) may also be needed in the functions dealing with splitting or concatenating the script queue. Suggestions are welcome either here as comments or on trac as enhancements tickets.
Update: added do_action('admin_enqueue_scripts', $hook_suffix); allowing plugins to easily queue scripts on the exact pages. To find out the value for $hook_suffix on a specific page, echo it from your function and visit the page.
Queueing a script for the footer follows the same syntax as in script loader:
$scripts->add( 'handle', 'full/url/to/file.js', array( 'dependency' ), 'version' ); $scripts->add_data( 'handle', 'group', 1 );
Group 1 means queue for the footer, group 0 (zero) is the default.
Hmm, any idea how do I queue script for the footer?
Just updated the post with an example.
Thanks =D What about the footer for the normal pages?
perhaps modify wp_enqueue_script to accept 1 more argument which is group?
So that one can do this
wp_enqueue_script('wp-postratings', plugins_url('wp-postratings/postratings-js.js'), array('jquery'), '1.50', 1);
I managed to enqueue stylesheets to specific pages in WP-Admin with the following code:
[sourcecode language="php"]
### Function: Enqueue Ratings Stylesheet In WP-Admin
add_action(‘admin_enqueue_scripts’, ‘ratings_stylesheets_admin’);
function ratings_stylesheets_admin($hook_suffix) {
$postratings_admin_pages = array(‘wp-postratings/postratings-manager.php’, ‘wp-postratings/postratings-options.php’, ‘wp-postratings/postratings-templates.php’, ‘wp-postratings/postratings-uninstall.php’);
if(in_array($hook_suffix, $postratings_admin_pages)) {
wp_enqueue_style(‘wp-postratings-admin’, plugins_url(‘wp-postratings/postratings-admin-css.css’), false, ‘1.50′, ‘all’);
}
}[/sourcecode]
As stylesheet will always be in the head, this will work beautifully.
Most people register JavaScripts via wp_enqueue_script() or a combination of wp_register_script() and wp_print_scripts(). Accessing $wp_scripts and setting the group via add_data() is a little bit weird at least from a plugin author point of view.
If we can get wp_enqueue_script() to have a “group” argument. The script loader will know whether when the JavaScript will be loaded at the header or footer and load it accordingly while still hooking onto “wp_print_styles” or even “init”.
In order to print JavaScript onto a normal page (outside WP-Admin), currently I have to hooked onto “wp_footer”, calling wp_register_script() and then calling wp_print_scripts().
Agreed, will add the extra param to wp_enqueue_script() and wp_register_script().
The front-side problem is a harder one. Many themes don’t seem to have the “wp_footer” call. The “get_footer” action can be used instead but that makes printing scripts there less effective as it runs too early, before the footer HTML. Even seen a few themes that don’t have a consistent footer and don’t use “get_footer”.
Thanks, i wait so long for this function.
@Andrew: Yea, some themes does not even have wp_head() in their header.php as well. The only way around this is to bring awareness I think.
Hello Andrew,
i hope you write over the new paramters for hook in the footer of frontend. This is very nice for better perfromance and control the js in footer. I like the wp_enqueue_* functions and i hope more themes-developer use thsi for intecrate the js-libraries. I see so much themes with a lot of plugins and many plugins load the sam js-library in the head and the site is slow and the user can not read the content. T think, wp_enqueue* is a very nice function for control js and css in the site. Very good for backend in plugins and also for hook in frontend.
*sorry for my bad english
I hope this optimization work without errors in version 2.8.
After more tests and research it seems the best two options for the WordPress admin are either minifying all scripts and loading most in the footer or concatenating and compressing them on per page basis.
Minifying and loading scripts in the footer gives slightly slower performance with both cold and primed cache and would depend on the ability to set appropriate caching headers on the server. On the plus side this method would be compatible with the current plugins and wouldn’t introduce higher server load.
On the down side the loading speed improvement with primed cache would depend on the availability of mod_headers and/or mod_expires (presuming most installations are on Apache). It seems many hosts have either one or both modules installed but that still leaves a lot of installations without proper caching headers. In these cases the browser would keep checking for updated content which would increase the loading time with primed cache considerably.
Concatenating and compressing all scripts would give better speed improvement and we will be able to set all needed headers. The last couple of days I’ve been testing a method that uses a separate php file similar to how the Gears manifest is produced. This is the same basic method used by many “website php compressors”, a stand-alone php function that gets as argument the names of the needed scripts then concatenates and compresses them. It doesn’t use server side caching (that proved to be problematic on some servers) since the cold cache page hits on the admin are relatively few.
The advantage is that WordPress is not run second time on every cold cache page load so it uses a lot less server CPU time and memory. The disadvantage is that it would work only for the default scripts whose paths are included in the script loader. This also seems to be compatible with all existing plugins as any additional scripts are loaded after the single script and all requirements are satisfied.
Another disadvantage is that a few hosts seem to compress all php output in a non-standard way that may result in double compression. This method would also need “Optimization options” screen with a few checkboxes that would allow the user to enable/disable the concatenating and compression as it won’t be needed when using Gears.
WordPress is using more and more JavaScript. A lot of features use AJAX and nearly all UI customizations and enhancements depend on it. In 2.7 there is some optimization for caching, especially when Turbo (Gears) is used all pages load in under 1 sec. However the cache control was left to the server and there seem to be a lot of servers with less than perfect settings for it.
There are many ways to optimize script loading in 2.8. Listing several of them roughly in order of ease of implementation/most effective:
In terms of performance the concatenating and compressing of all js and css files for each page gives the best results, closely followed by the loading of most js files in the footer. If we decide not to do the concatenating/compressing, we should revisit it in about a year (for WordPress 3.0?) as most Web hosts would have upgraded to php 5 by then. The actual php version wouldn’t affect this, but it seems that while upgrading the hosts also improve their server and php configurations too.
It would be great if wordpress was compatable with mod_concat and mod_jsmin. I have attempted to implement both apache modules with 2.7 with limited results
If hosting servers don’t support PHP5 now, then they’ll probably won’t in another year or two.
mainly now all hosting servers support php5.
Many of your ruminations are understandably taking into account the huge array of available WP plugins. While this is absolutely necessary, maybe taking a bit of a different tack can open up some options?
I have often wished that I could manage the load order of plugins and control which loads before another, or even WHEN a plugin loads at all. Could it be that implementing more internal plugin declaration and control could help with the overall compatibility of this optimization initiative?
If we make it possible for the system to detect the way certain plugin scenarios will affect optimization efforts, then the system can adapt on its own. Rather than making global decisions on optimization based upon what COULD exist with certain plugins, maybe taking the tack of allowing the plugins to tell the system what they load and require would do the trick?
The plugin header could potentially be extended to either allow the developer to state when the plugin should be loaded, or at least to specify other items that it either requires or loads itself. A “Loads:” field along with a “Requires:” field could go a long way to helping WP understand what the activated plugins require. It could also help the user make intelligent decisions by allowing the system to report if the activation of a specific plugin will necessitate the disabling of a specific optimization technique that would otherwise be used by the core.
Alternatively (or in addition to the above), you could allow the WP admin to specify where and when all active plugins are loaded. While I would love to see this type of completely configurable plugin management, I think that even just the additional declarations could be extremely helpful.
Sorry – I don’t mean to digress from optimization to plugin management, but it seems that there may be a few strong ties between the two.
Thanks for listening!
Cheers,
A
@Jacob Santos, @Rahul Boss yes, but most hosts require the user to manually switch to php 5, sometimes by directly editing .htaccess. Don’t think that most users are comfortable doing that or even aware of it.
@miloandrew almost all of these features already exist in the API. In the case of script loading, plugins can enqueue both js and css and require any of the scripts included in WordPress. Unfortunately not all plugins use the script loader, but most do.
The optimization will have to be transparent for the plugins. For example setting appropriate caching headers for js, css and images in .htaccess would affect all scripts.
The load order can be managed from the filter/action priority quite well. The blog admin can edit the plugin’s file and change that if needed or have a small plugin that would set it up for the rest.
BTW the first step is already implemented. All js has been minified reducing the size by about 30% – 35%.
I am not sure where to post this, but I’ll start from here since title of this post describes my intention.
When new comment threading for 2.7 was introduced, you recommended call to comment-reply.js in this form.
But problem with this is that comment-reply.js is called on all pages in all cases even there are many cases when it shouldn’t be called because it is simply not needed:
- when comments are turned off on site wide level
- when threaded comments are not enabled (this is default option)
- when comments are not enabled on that entry
- when there is no comments on that entry
- when page is used as front page (in reading options)
In all those situations comment-reply.js is loaded even though there is no need for it. Yes, it is cached after first page view, but what if visitor lands on that particular page that doesn’t need this file and that page already has ten javascript files? It will just increase page load time and you know that visitors that come from search engines or from links on other sites have low tolerance and don’t want to wait to much.
So what we need is that this script is loaded only when there is need for it and my suggestion is that you make some function that should be checked and that will return true or false value when asked if there is need for comment-reply.js. Maybe that script can’t solve all problems but at least it can return false value if comments are globally turned off or comment threading is not enabled (I’m quite sure it can do this).
Example of how this should work in theme’s files:
if ( comment_threading_entry_status() ) wp_enqueue_script( 'comment-reply' );
Example of functions:
function comment_threading_entry_status() {
if (((get_option('default_comment_status')) == "open") && ((get_option('thread_comments')) == "1") && ( !is_front_page() ) && ( comments_open() ) && ( have_comments() )) {
return true;
} else
return false;
}
This function doesn’t work like this (it works without last two conditions), but you got a point.
I hope that you’ll at least respond me in reasonable time
(btw, why we don’t have subscribe to comments here, it should be enabled on wp.com?)
This is a great idea. The problem is even bigger than you describe. It’s true that for second and subsequent pageloads the javascript is is the cache, it’s also true that unless the files has an expired header the browser still has to contact the host to determine if the file in cache is current. There won’t be a download for current files, but the round trip to the server just to check the currentness wastes time for every page. This problem exists for losts of files in lots of plugins and probably contributes to a 1 or 2 second page delay for the average page.
Denis de Bernardy 8:08 pm on April 1, 2009 Permalink |
Isn’t there a way to add an svn hook on tag?
http://svnbook.red-bean.com/en/1.4/svn.reposadmin.create.html#svn.reposadmin.create.hooks
That way, branches would also be pretty formatted, and only actual released versions would be minified.
D.
DD32 10:04 pm on April 1, 2009 Permalink |
Its a thought, But many run on trunk daily anyway.
I cant remember what the constant is to use non-compressed scripts, Maybe if WP_DEBUG is enabled, then non-compressed scripts could be used instantly?
Xavier 8:37 am on April 2, 2009 Permalink |
Sounds like a must to me.
Also, why not go one step further? Keep trunk’s PHP code clean and commented, and minimize that too in the release branche.
Of course, that would mean those why want to explorer WP’s inner-working would have to grab the trunk, and that older full-version of WP wouldn’t be fully available.
So (typing while thinking here), would it be cool to cater to everyone : provide a fully-minimized archive (PHP, CSS, JS) for default users, and keep a “maximized” archive as-well, for further reference?
(unless, it’s all part of an elaborate April-fools scheme, in which case, well, my case still holds
)
Gmcosta 5:45 pm on April 2, 2009 Permalink |
too much work I suppose.
Peter Westwood 6:20 pm on April 7, 2009 Permalink |
Minimising too much in the branch/tag will just make patching / generating patches too much work.
I can see the benefit of js/css minification but not php.
Xavier 2:50 pm on April 10, 2009 Permalink |
@Peter: obviously, and likewise for CSS and JS, at a lesser level. Hence my suggestion for two archives per tag: a fully-minimized as default (for those who won’t look into the code), and an untouched one (for devs and the curious at heart).
I’m not suggesting minimizing in the branche, since that’s the basis for any further minor version, but for the the tag, which is meant to be definitive – or so I understand.
I could be way off, obviously. And the profit could me meaningless, but hey, we’re here to discuss
Robert Accettura 12:59 pm on April 2, 2009 Permalink |
IMHO this should be part of tagging. There’s a few advantages here:
1. Less overhead for last minute changes on branch.
2. Most people who want optimization and polish and can’t do it themselves run tags anyway.
3. minimized code is harder to debug. If your not using a tag, you should have easy debug abilities.
Why not just automate the creation of a tag with a make/ant/shell/configure/whatever script that creates minimized versions of js/css (use the format [name].min.[ext]) when a tag is made and changes a flag in wp-config.php to prefer the .min if it exists. This has a few nice things:
1. Tagging is as simple as running a script which will create the tag, minify, and commit.
2. No worries about forgetting to update minified and non-minified code like on a branch since tags are (or should be) final.
3. By using a flag in wp-config.php to prefer minified version of js/css you can just change the flag to use the non-minified versions… This is awesome for when you want to debug.
4. Theme/plugin authors can utilize the wp-config flag to ensure their theme/plugin offer a minified/non-minified verisons.
5. As a bonus, you could include the minifier in WP to make it even easier when editing css/js via wp-admin.
Just my $0.02
Nate Moore 9:18 am on April 12, 2009 Permalink |
I agree
Simon Wheatley 1:53 pm on April 18, 2009 Permalink |
How about having the JS in two states: uncompressed (which the default enqueuing stuff uses if a constant, maybe reuse WPDEBUG?, is specified in wp-config.php) and compressed which is used by default. Uncompressed is used for dev, and an SVN post-commit hook script could automagically compress/minify/pack/whatever’s-currently-best the committed uncompressed JS files. An SVN ignore property could be set to ignore all *.cmp.js files (or some other filenaming schema for compressed files) to ensure they don’t get committed accidentally.
Peter Westwood 5:58 pm on April 18, 2009 Permalink |
I believe this is sort of what Andrews code in trunk already does.
Look at the stuff in script-loader.php in wp-includes.
Shane 12:45 am on April 19, 2009 Permalink |
I am kinda liking this idea.
Andrew Ozz 1:54 am on April 20, 2009 Permalink |
Peter is right, that’s almost exactly how it works in trunk now. The switch to turn on .dev.js loading is SCRIPT_DEBUG.
Andrew Ozz 1:51 am on April 20, 2009 Permalink |
Currently both the minimized (.js) and the non-minimized (.dev.js) versions of all custom scripts are included. This was mainly so we can test if minimizing works well. If we decide to go with two builds per release: “production” and “development”, we can remove the non-minimized scripts form the first (will have to change script-loader there too). This will reduce the download size/number of files in the install package a little bit, not sure if it’s worth it…
Stripping all PHPDoc/PHP comments would reduce the download size a lot more.