Wrote post for dev blog per last dev chat about setting scope for 3.0: http://wordpress.org/development/2009/12/setting-scope/
Also created a forum thread to start the discussion of which features might be worth putting in 3.0. http://wordpress.org/support/topic/345127
Trac Christmas Present for all WP developers: I changed how the comments work so it now links the username to profiles.wordpress.org and of course shows their Gravatar. 
http://core.trac.wordpress.org/
Working with the templates there is understandably horrible (for future reference, /usr/local/src/working/Trac-0.11.6/trac/ticket/templates/ticket.html is core with mods going in trac/trac-environs/wp-core/site.html) but we can make more mods like this in the future. What else should be linked, and where?
Since in the last dev chat we agreed to take the holidays off and come back in January and determine the scope for 3.0, I’d like to suggest that people stop changing ticket milestones to 3.0 for now, so that we can reduce the amount of Trac rehashing we have to do after we set the scope in the first dev meeting back. Also, we may want to plan on that first meeting being longer than the usual hour.
There are already 320 tickets against the 3.0 milestone without even thinking about the merge, so if anyone is antsy to work on stuff for 3.0, those tickets are probably a better place to start than something that got punted to future release instead of 3.0.
Maybe the 3.1.0 milestone should be created and the tickets/features that are not as big as lets say:
*WPMY
*Media Overhaul – This should be done at the same time since there will be lots of code changing and what not.
I would even maybe move my widgets API idea off till 3.1, but if put in 3.0 the long dev cycle should help with the transition with most widget authors if we adopt the idea.
I am thinking we need at least 2 hours to discuss the huge changes for the 3.x milestone.
The 2.9 beta in a nutshell: When I find bugs from older versions, I’m told they are “late” for consideration and get punted. When I find bugs in 2.9, I’m told they are invalid because the patch hasn’t been written yet. When I submit a patch, it gets punted. If I find some trivial mistake in the existing syntax, my patch is quickly applied and closed out. Half the time my code is changed without discussion, thus creating new bugs that I was careful to avoid when I wrote patch. It’s been an interesting (awful) experience trying to participate in this project; one that I will not be repeating.
Care to cite your sources? I have a hard time believing this is the case. While enhancements and minor bugs are indeed punted as there isn’t time to properly test those, deal breaking bugs are fixed.
For me as a webmaster, “deal breaking” means any bug that prevents me from upgrading, or that requires me to hack the code base before upgrading. For the dev team at large, “deal breaking” seems to mean that unless the software crashes and burns, the beta version is going to be released unmodified. You are welcome to look at the reported by miqrogroove list in Trac to see all of my un-fix-worthy bug reports. (and the lucky few that made it)
The tickets in question broken down by resolution.
Particularly, those under “Resolution: None” have all been punted for one reason or another. Most of them are holding up testing or other known bugs that I haven’t bothered to report due to dependencies.
There seems to be a disconnect between what some people consider major bugs and what other people consider acceptable known edge-case or low priority bugs. This happens in software development when there’s not a central agreement on what constitutes various severity levels. To prevent this from happening in the future, I suggest that before we get into 3.0 we write up a list of our severity/priority levels and define them once and for all. Then, if someone doesn’t agree that a ticket should have been punted (for example, reporting an IE6-only slow load of a particular flash popup screen when we are already in beta), they can be pointed to the published severity/priority definitions to better understand the thresholds of importance on bug fixing.
That’s going to need some refinement. If you do it that way, it’s going to come off sounding like “We are setting the bar so low for QA on the next beta, that our policy is to reject all valid patches with only the following exceptions…”
Let me also draw your attention to some of the Fixed bugs. http://core.trac.wordpress.org/ticket/11257 Why did I have to kick and scream to get (half of) my patch applied for a bug that makes blog pages blank out? That’s what you call an “acceptable known edge-case or low priority bugs”? Is it reasonable for me to be *amazed* that any action was taken on that issue?
I too feel that the current situation is rather as miqrogroove described it. Hopefully it can be sorted out before 3.0, but I’m not holding my breath…
I agree that this policy definition mentioned by Jane is sorely needed, but I feel that unless it is decided by the community rather than just the core devs, it will only serve to reduce discussion and drive away more contributors, and not to actually improve the situation.
The severity/priority level list should be decided by a public poll or survey, or it will be useless.
@caesarsgrunt: there actually is a vote +1 / -1 link to the top right of ticket screens. Few people give it the slightest attention, however.
@Denis: Yes, I know – I didn’t mean users should vote on tickets, I meant the policy regarding what can be committed at what point in the release schedule should be decided by a community process involving a survey.
@caesarsgrunt: I do agree we need as a community to agree on the scope of what changes should be made when. To this end I have tried to summarise my feelings on the subject and opened up the discussion here – http://wp.me/p40k-2a
Hopefully the community can have a meaningful discussion in the comments on that post and then we can have a well understood process.
Here’s another beautiful one:
@westi: agreed in theory, and replied to your post. But we’re nearing the stage (being positive in nature, I’ll make the assumption that we already haven’t gone past it) where the question no longer is which bugs gets fixed or not. Rather it is whether potential contributors decide whether it is even worth their time to report bugs and contribute patch.
Please take this remark, not from me as a regular ranter who continues on to think that we need to double the number of core devs; but from me as a regular bug reporter (try to find a single one with more bug reports; you’ll find it very hard to do so) and patch contributor.
I had to object to the premise of that article. It sounds like “in maintenance releases, we only fix catastrophic bugs.” That’s the cause of the problem, not the solution. When you fail to incrementally patch bugs across the board, you end up with massive, late upgrades that are hard to test.
Oddly enough cron seems broken for me too.
Hi folks. I’m in the process of putting the 2.9 release video together. I’ve just posted a WIP over on the 2.9 page of the codex. If you have any thoughts on improvements, additions, things to remove, or any other suggestions I’d love to hear about them in the comments here. Preferably in the next… oh, 16 or 17 hours
Thanks in advance!
Nit picking time!
The URL-on-it’s-own-line-turning-into-an-embed is enabled by default.
I’d move DailyMotion further up the list. I hear it’s fairly popular in Europe (France especially), so worth mentioning early.
Cool – will “patch” accordingly. Nit picking most welcome!
“So for example [...]” → “For example [...]” Three sentences start with “So” and this one works least well.
Well spotted – one of my many crimes against grammar. Fix in the works
Also that MySQL min requirement is at 4.1.2 for stand-alone installations.
Thanks Shane. I think it might be worth pointing viewers to the technical requirements directly, rather than trying to fit them into the brief overview. It has the danger of slightly breaking the flow if we get into too many of the finer points. On average the release videos bring in 1million+ views, and I’m not sure what percentage of those are people with the technical knowledge to appreciate the technical details, however important they are. I’d opt for adding a pointer to the technical requirements in there to cover this one. Anyone else have any thoughts?
Pointing out the SQL change is a big deal, even from a non-technical perspective. MySQL 4.0 is a dinosaur whose database language is incompatible with all newer versions. Except for the simplest of code, this new 4.1 requirement lifts a major hindrance to compatibility and efficient development. It’s like the PHP5 of the database world. It’s great for developers. It’s a mixed blessing for anyone running MySQL 4.0.
@microgroove – I understand your concerns, but would still contend that an overview “headlines” type release video isn’t the place to list technical specifications or issue installation caveats & warnings. If the MySQL requirement update does prove to be an important issue I’m sure it will be noted in the installation documentation and on the download page, which seem like more appropriate places.
That said, I did add a pointer to the technical specifications in the final version (now nearing the end of production) in addition to further reference to the many fixes and additions in 2.9 that you mentioned below. You can see the final script, including these ammendments, over on the 2.9 Codex page. Thanks for your input!
It looks great for a video about new features. Unfortunately, those new features pale in comparison to the list of bugs that went unresolved until version 2.9. Some of my favorites:
#9471 All media uploads were orphaned unless “Save Draft” came first.
#10004 Auto-saved posts had wrong username.
#10326 Google rejected feeds due to UTF-8 corruption.
#11129 Dashboard configuration broken in Internet Explorer.
#11289 Logout caused Internal Server Error.
Fixing those is a great accomplishment, of course. You might consider mentioning that several major bugs were fixed in the Media department. It sounds a bit more reassuring than, “We’re adding all sorts of doodads on top of what you already have.”
I agree, miqrogroove, that a lot of the most important changes version to version are the less glamorous (but totally essential) fixes. The key thing with the video is to give a very brief, blog-embed friendly overview that complements the richer details in the release post, and what a lot of blogger’s choose to write about is “what’s new”. That said, I think more of a hat tip to the work here is definitely worth adding, even if it means pointing out to the finer details elsewhere. I’ll have a crack at working this in. Thanks!
Suggest Agenda Items for Dec 17th Dev Chat
Disclosure of api.wordpress.org data transfer/retention policy: consider adding in some conspicuous location (e.g. options-privacy.php) the following disclosures:
1) State that core update notification requires sending WordPress version number to api.wordpress
2) State that plugin update notification requires sending list of plugins with version numbers to api.wordpress
3) State that theme update notification requires sending list of themes with version numbers to api.wordpress
4) State how often api.wordpress is queried, what additional data are sent to api.wordpress, what data are retained by api.wordpress, how long those data are retained, and what is done with those retained data.
What I’d like to see instead, personally, is a page on wp.org that actually lists part or all of the information (minus the urls, for obvious security reasons).
Agreed, the intention was always to expose as much aggregate data as possible. It could also be useful for plugin authors.
Well, in case you haven’t written such a page yet, you’re welcome to reveal the schema used, as well as the existing scripts that take advantage of it. Surely someone will pick it up and write part or all whatever is missing to distribute it in the wild.
The data is retained forever. In addition to every theme and plugin, we’ve developed an algorithm to figure out what blood type you are by the PHP and MySQL version. Plugin data has proved surprisingly useful when paired with spidering your blog posts because by the TV shows you talk about we guess your age and then whether you say “soda” or “pop” tells us your location and bing we have your social security number.
That makes the data super easy and profitable to sell, which we do. Or rather, did. The main customers buying the plugin/SSN information were some insurance company AIG and Lehman Brothers (sp?) who used it to tweak credit tranches but… apparently that didn’t work or something? Anyway congrats to all the Akismet and All-in-one-SEO users who bought a house last year!
The not-so-secret plan (I talked about it at WordCamp SF!) is to use the data to predict which plugins are going to become popular before they become popular so we can replace the core team including myself with a script that just auto-merges plugins into core (and removes whitespace from the end of lines).
Oh, and don’t worry about the organ donor checkbox in 3.0, my liver is probably going to last at least until I’m 28.
Ouch. This response is a Public Relations nightmare. Those bringing up the privacy issues I believe have legitimate reasons for doing so. Shutting down the noisy minority like this is pretty bad. Wouldn’t it have been better to just answer the questions posed with regards to Chips suggestions, especially number 4. I mean, if the types of data, data retention period, and how the data is used were all added to the privacy policy, that would be a step in the right direction. I’m also for telling users up front when they install the software the privacy policy. They might not read it and skip it but then they have no reason to complain.
I don’t think it’s a good idea to opt-out of any updates. Rather, I would hope that opting out just provides a way to send less data such as blog url and such but still be able to do upgrades.
I hope the other WordPress enthusiast sites don’t pick up on this or there is going to be another negativity ****storm.
I’m beginning to wonder when WordPress.org will hire a PR firm lol.
I’m afraid I have to agree with Jeffro. Although personally I have no problem with either the data sent or data retention (I really don’t care either way) Chip and others have brought up a valid point and responses such as this leave them feeling as if their concerns/opinions are stupid and worthless.
Lol. Amusing response.
It might pay to add a disclaimer saying you still take their suggestion seriously on the end of stuff like that in future as it could come back and bite you in the rear at some point otherwise. Those speaking up about this topic seem to be quite aggressive about it and they may not see the funny side.
Yes, it was meant as a satire to lighten up the discussion a bit, people seem to be getting a bit serious.
As to Chip’s specific question about what data is stored, I took a look and it looks like we currently store the latest update sent but no historical info. However I do want to keep historical aggregates for everything so we can tell plugin and theme authors how many people actually use their stuff over time and we can get away from using easily-spammed downloads as a metric for popularity. I believe this is fully covered under the Aggregated Statistics section of the WP privacy policy.
While I don’t really care what info you farm, store, and use, responding in this way to people who have a valid concern may provoke them to fork WordPress into something Automattic doesn’t use as a data-farming operation. And perhaps this is the solution: a branch of WordPress without any connection to Automattic, by api call, ownership, direction or input. Would the project survive that? And all this over an issue that has such an easy solution.
This type of spiteful rebuttal lends credibility to those who have a concern for how this data is used. Who does own that data, and the rights to that data? Is WordPress.org equal to Automattic? Does Automattic have the right to access, without permission or reservation, non-published database information from WordPress run blogs too? How about credit-card and social security numbers, and form emails processed by plugins? Are you worried that how you are really using the data is in fact harmful if publicly known? Is making a disclosure a liability for Automattic? Does some other 3rd party pay a lot for that information?
Why provoke people just for emotional gratification? The solution isn’t heinous. Disclose what information, how its gathered, how its used, and why its necessary. It’d be nice to have an option for not participating in non-essential data, but not absolutely required, as long as its disclosed what this info is, why its collected, and how it would be used, etc. Besides, next to no one would opt-out I believe.
Actually Ken, that project wouldn’t survive. Threats of forking isn’t going to change anything and neuters any momentum behind the privacy argument you’re trying to make here.
For those of you who haven’t read it already and don’t agree with Chip’s stance, I suggest reading through the following trawl of posts:
http://www.wptavern.com/forum/general-wordpress/1085-wordpress-phone-home.html
That topic is overly long, but there are some good points raised in there which have solid arguments for and against Chip’s suggestion.
This is paranoia by those worried about it for their own site. But I’m guessing there is probably a legitimate legal concern too since WordPress.org is storing potentially private data without permission (and yes, some of it is private – read through the forum topic for a better explanation).
In my case, there’s no paranoia in regards to my own site. There is concern about people disabling update checks and also concern about possible legal implications if any clients try to make consultants responsible for not disclosing the risk to private information. We can’t even give reassurances because nobody is giving them to us.
Please see my response at http://www.wptavern.com/forum/general-wordpress/1085-wordpress-phone-home-19.html#post9628 and see whether some of that “paranoia” isn’t justified.
Disclosure somewhere is enough to address the issue. Opting out can be achieved by not using WordPress, or finding/making a plugin. An option in the core is a bad/unnecessary thing. As long as the info is provided, and the user makes an informed decision, there is no problem, no controversy.
First, not disclosure “somewhere”, but rather, disclosure *within WordPress* (thus, my suggestion of options-privacy.php).
Second, how can a user make a decision, when that decision (opt-in) is made for the user, with no ability to *change* (opt-out) that decision?
One can “opt-out” of data transmission by “not using WordPress”? Seriously? How asinine.
Or, finding/making a plugin? No. The data transmission is built into WordPress, therefore so must be the mechanism to opt-out (again, I am referring to data transmitted for the purpose of *retention*, not for the purpose of update-checking).
With respect to data *retention*, to say that “an option in the core is a bad/unnecessary thing” demonstrates capricious disregard for the users’ right to control their own data.
Again, I direct you to stopbadware.org guidelines for transmission of user data to a third party.
Look at Mozilla’s privacy policy for Firefox: all data transmission is fully disclosed. No data are retained for update checks. A means to opt-out is provided.
Why should WordPress be any different?
I just spent 5 minutes clicking around Firefox and I can’t find a privacy policy anywhere within the application, nor can I for any of my extensions. Of course on their website it’s easy to find, just like it is on ours.
To find the Firefox Privacy Policy (from within Firefox):
Help -> About Mozilla Firefox -> Licensing Information (or, open about:license) -> Know Your Rights (or, open about:rights) -> Privacy Policy (link)
Rather a convoluted process. And the policy is only on their site. Would a “Privacy policy” link in an obscure wp-admin page (we were thinking of a “credits” page) be an acceptable solution? It would be equivalent to Firefox’s communication of their privacy policy.
Oh, I agree, it’s pretty convoluted. I would prefer a more conspicuous location, but I think the implementation is fine. Something along the lines of:
“WordPress sends certain user data to api.wordpress.org. See the WordPress Privacy Policy for more information.”
p.s. I’m working on a draft. What should I do with it? Put it up on Trac?
So, here’s my attempt at a draft Privacy Policy for WordPress. Thoughts?
What I’m saying is enabling a feature in the core to disable updates is “asinine”. Whatever data is necessary for updates should NOT be optional, to be enable/disabled as an option in the core. It should be explain what and why this is. My comment was in support of your numbered list (the above one about disclosure, not the below one about opt-out).
Non-essential data should be optional. What I was attempting to say, is that disclosing what info is involved here solves the problem. The problem is gathering info without knowledge, and therefor consent. There is no requirement to be able to turn off data transmission because that is technically optional by use or not use, as long as it is made known that use includes such communication. This observation is not “asinine”, its a reality, not matter how distasteful it seems. Your tone is unappreciated. This is the area that seems related to “PR”; that Automattic, a private company, uses data collected from an open-source community project. I believe it is a valid concern. And yet the primary issue can be solved separately with disclosure.
“First” “Somewhere” does not preclude options-privacy.php, and if you heart is so set on that, then I wish you the best.
“Second”, if informed of what’s involved, users have a choice in whether or not to participate, and that is the concern here. Not using WordPress is a valid option, not “asinine”. Stating otherwise severely weakens what position your argument has. As far as your numbered list, the only optional information is in number 4 (retention). My statement is that it’s merely unnecessary to solve the problem. I’ve said else where I support a choice on participating in optional data, and agree about data retention, but upgrading should not be optional. It’s still irrelevant to the main concern here.
I was primarily responding to the thread this comment is in. The one with “disclosures” as the qualifier for the 1-4 list. Your list below is the opt-out list 1-4 of which I am vehemently opposed to 1, 2 and 3. I can only agree with the 4th position, while I am for 1, 2, 3, and 4 above (again, I’m for disclosure 1-4, and against opt-out for points 1-3, while 4 is fine with me).
So, in summary, opting out of “api.wordpress data retention” would be a desirable option, disclosure in my opinion is all that is absolutely required here.
“Somewhere” could be taken as http://www.wordpress.org, or essentially, anywhere. My point was that the disclosure needs to be made *to the user*, and therefore, *within WordPress*. Whether that’s during installation, or on the Privacy Options page, or wherever is fine, provided that it’s *within WordPress*.
And, yes, I do find the assertion that “if you want to use WordPress, your data *will* be transmitted to and retained by api.wordpress; if you don’t like it, don’t use WordPress” to be asinine. It’s along the same user-unfriendly lines of, “if you don’t like it, fork.”
However, it seems that we’re in agreement on data retention being optional, and I’m not wanting to engage the battle on opting-out of core/theme/plugin updates, having seen the comments here. I already conceded those points. So, I think we’re in agreement on the 4 opt-out points below.
(And for the record, even if an option to opt-out of update checks were put in, I wouldn’t set it. I like update checks.)
I pretty much agree that, for the most part, disclosure will solve the problem.
“So, in summary, opting out of “api.wordpress data retention” would be a desirable option, disclosure in my opinion is all that is absolutely required here.”
If we had those two: full disclosure and a mechanism to opt-out of data retention, I think the entire matter would go away.
Opt-out option for api.wordpress data transfer/retention. On the Privacy Options page (options-privacy):
1) Allow users to opt-out of core updates
2) Allow users to opt-out of plugin updates
3) Allow users to opt-out of theme updates
4) Allow users to opt-out of api.wordpress data retention
I think you’re beating a dead horse. Users should not be allowed to opt out of updates without a plugin. Period. Those that do can and usually do get hacked, just like they deserve.
Re the data that is send, I think you’re misunderstanding how the data is used in the first place. Some plugins and themes have identical names. At least the file, author, and url are needed. Working around this issue would require all themes and plugins stored on wp.org to add a unique identifier. That would be a carnage.
Seriously, the only point that makes any sense in all of this is removing the site url. But even for this, I can see compelling arguments to keep it — in particular if it’s used to weed out spammy plugins used by spammy blogs that then send ping notifications to pingomatic.
Hey, I’m just throwing it out there for consideration.
As for allowing or not allowing opting-out of core update checks in core, I’m open to that argument. That’s why I split the four out into separate points. I don’t buy it quite so much for plugins and themes, however.
On the other hand, an opt-out option for data *retention* (as opposed to sending data needed for update checks) simply *must* be included in core. I find no compelling reason presented thus far, against such an option.
And, if opting-out of data retention were entirely separated from sending data necessary for update checks, I imagine that very few people would have reason not to let WordPress send data for update checks – thus eliminating the security risk associated with disabling update checks.
I see no reason why it “must” be included. The data is stored and used to determine what minimum version of PHP we should support and the URL is used as the unique identifier.
It’s not like it’s your SSN being sent. It’s just a URL and some server info.
It *must* be included because it is the right thing to do. WordPress – just like every other application (be it from Microsoft, Google, Apple, Ubuntu, Sony, Nintendo, or anyone else) must respect users’ right to control of and privacy with respect to their data.
The data being sent by WordPress does not belong to api.wordpress; it belongs to the user. Even short of a legal requirement (which, in some cases, there may be), it is the ethical, respectful thing to do.
“Users should not be allowed to opt out of updates without a plugin. Period.”
+1
@Chip: Plugins and themes are just as capable of introducing security risks that need to be closed with updates.
Again, I am most concerned about users’ right to opt-out of data *retention*.
I am fundamentally opposed to not having opt-out options for all of the above (because, as the owner of the application, the user should have this control), but I’m not wanting to engage in that fight. I suggested them for discussion, but can see where that will go.
So, conceding the update-check opt-out option in core, what of the data-retention opt-out option in core?
@Jane – +1 BUT private, (ie. custom code that is not released to the public or not hosted by wordpress.org) plugin and theme data is being collected. This is nobody’s business except the site owners and should not be reported back to WordPress.
At the moment, if anyone wants to keep private plugin and theme data private, or if they don’t want their URL associated with the data being collected the only option is to disable update checks completely. If privacy concerns are resulting in deactivating a core feature and WordPress wants users to keep update checks then perhaps it is wise to remove the obstacle.
Giving users a reason, regardless of whether anyone else feels its a valid reason or not, to disable update checks defeats the purpose of providing them.
I agree that users should not be able to opt out of update checks without a plugin. But they should be able to control what information is sent and restrict the sending of personal information if they wish or if laws in their own countries require this.
If WordPress is all about freedom then it should also be about users having freedom to control their own personal information.
That’s UI bloat for sure.
I think a constant in wp-config.php would be much better. Can’t be changed by a hacker who compromises the login and allows blocking before the site is even installed. Then again I’ve never understood these paranoid people who want this stuff blocked.
Is it “UI bloat” for every other application in existence, that must disclose to users when data are sent to a third-party server, and that must provide an opt-out mechanism?
Why is WordPress exempt?
And every time I hear someone call people with legitimate concerns “paranoid”, I am going to assume that the speaker is acting arrogant and condescending.
There is no need to start throwing around insults. Please remember that your opinion is just that — an opinion. Someone is not wrong just because they disagree with you. I for example don’t consider it a legitimate concern.
Anyway, I’m stepping out of this conversation.
Indeed.
And incessantly referring to people who express concerns as “paranoid” is no less insulting than calling those who dismiss those concerns “arrogant” or “condescending”. It’s not directed specifically at your comment. I’ve seen that term thrown around here, and on wp-hackers, and at WPTavern.
Mutual respect gos a lot farther.
To clarify, Chip, I think any and all privacy concerns are completely valid to the person making them. I’m passionate about privacy myself, for example I Truecrypt encrypt all my drives.
Fortunately WordPress is completely open source and you can read every line of code that sends every bit of information, everywhere. (Not possible with most applications!) You can modify the code directly, or use or create a plugin to modify or prevent anything from being sent.
If you consider your blog URL to be sensitive information, which is exceedingly rare but certainly conceivable, but still want updates I would recommend hashing the URL before sending it. (And also disabling Ping-O-Matic updates.)
If you consider server IP to be sensitive information, which is exceedingly rare but certainly conceivable, I would recommend either disabling updates with a plugin or by modifying core per above or putting all external requests through some sort of proxy which can be done at the server level or perhaps through extending WP’s HTTP class.
For the truly cautious, many of these precautions are best handled at the server level, for example using an iptables firewall to block all outgoing connections from a server or anything on it, including WordPress.
We’re lucky that WordPress is software you can download and configure yourself because it gives you 100% control over every aspect of its operation, and you can run it under any environment you can imagine.
Matt, I greatly appreciate that WordPress is open source, and that everything “under the hood” is available.
That said, my own PHP skills are marginal at best – and, I would imagine, far above that of the average user. I do not believe it is sufficient to expect an average user to figure out what data are sent to api.wordpress only by looking through the PHP files.
Further, knowing what is *sent* to api.wordpress in no way provides insight into what is *retained* by api.wordpress.
And, again, the nature or sensitivity of the data being sent is irrelevant. What is relevant is that the *user* owns the data, and therefore WordPress must still respect that ownership. I have linked (several times) the StopBadware.org guidelines for applications that transmit data to third parties – guidelines that require up-front disclosure, and an option to opt-out. This really isn’t about paranoia; it is about adopting industry standards for regard of user data.
(Thinking farther ahead, it is also about building trust and goodwill from users, so that you might in the future be able to have users opt-in to having even more data collected, that would prove to be even more useful.)
I honestly believe that almost everyone would be placated with up-front disclosure of the data transmission (both what is sent, and what is retained), and the ability to opt-out of data *retention*.
As for the other examples you have given (e.g. Ping-O-Matic): a) the user has complete control over sending data to POM, and b) POM doesn’t *retain* those data (AFAIK).
Besides, it is not the URL *alone* that would normally be considered sensitive or especially valuable, but rather the cross-reference of other, far more sensitive/valuable data that can be tied together by the URL.
I don’t particularly understand the push-back on the issue of disclosure. It would be incredibly simple. I’ll even offer to write something up (though I’m sure it would need to be quality-checked, as I’m sure I wouldn’t get everything right wrt the data retention, and it would probably need legal review). Regardless: we write something up, drop it on options-privacy.php (or wherever is most appropriate), and it’s done. Maybe point the user to the location at the end of installation. But that’s it. That’s all that’s needed.
I do understand, though, that it would require some work to separate update check data from historical/aggregated data, since much of the data are the same, and would have to be sent differently if the user is allowed to opt out of data retention.
I still think the goodwill/PR gains will far outweigh whatever developer time is required to make such a change.
Please don’t get me wrong. I LOVE WordPress. I want to be able to praise WordPress as being a leader on the issue of respecting user data, while also being able to use those data for the betterment of both WordPress and the WordPress community.
The argument that WordPress users concerned about privacy should seek out a plug-in is flawed in my opinion. That makes the assumption WordPress users are aware that private information is being sent in the first place which is unlikely the case. Second, in order to ensure privacy on future versions of WordPress, the plug-in would likely have to be maintained to ensure compatibility, would it not? If a compatible plug-in was not available say after a significant upgrade then privacy is once again not assured.
A notification screen viewed at time of WP install and a privacy document with the download package would be a good start.
People concerned about privacy are just fine today — there is nothing we send or do that compromises anyone’s privacy. The information we send is public and regardless we don’t publish anything potentially personally identifiable, like URL, we only publish and use the numbers in aggregate for purposes of improving the software and user experience. This is like how browsers send your IP address with every request, it’s appropriate for 99.99% of the audience and if you’re a political dissident or government employee where that is considered sensitive then you should use a proxy or anonymizer plugin.
Also Chip’s objections don’t seem to be with anything we send, it’s with the data being retained which isn’t explained anywhere except my comment above, which we can probably put somewhere. In January or February I’ll talk with our legal folks about a revision to the WP.org privacy policy.
Thank you Ryan.
Please discuss contacting the Software Freedom Law Center for advice (which is free) on what data is legal to collect, how, whether opt-in is necessary given the cross-jurisdictional nature of WordPress distribution, the nature and length of time for retention, whether data matching is permitted, and the type of disclosure that needs to be made.
http://www.softwarefreedom.org/
Discussion about these issues is now impossible. It was impossible in 2007 and every privacy concern that has been raised in the past two years has met with similar responses. The only practical way to put this issue to bed is for the team to obtain legal advice and act on it. The SFLC is free to projects like WordPress.
Agreed with Elpie on this.
This entire argument comes down to what you feel is private information and what you feel is not. As an IT guy, I don’t see any of the information that they send as “private”. IP addresses are public domain, so no point in hiding those.
Plus, as far as users opting out, why should the option even be allowed? I can’t opt out of the Federal government accessing my records of telephone calls, e-mail conversations, or anything to that regard? I can’t opt-out of Verizon Wireless knowing my location at all times? I can’t opt out of publix tracking my spending habits to know when to stock things?
Disclose how and what data is sent, sure. Just put a link to the privacy policy on WordPress. Beyond that, seriously? come on. Make a plugin if you feel that strongly about it.
These are realities one must realize in this digital age, and in all honesty, if you don’t like it, Don’t use WordPress. That’s not asinine to suggest, considering that if you don’t like it, find another cms that doesn’t.
Other ideas that might be worth discussing while on the topic of call home:
Yet another one on the same topic:
The update that sped up the opacity fade in/out on alerts has made it into wordpress.com. Wow, is it annoying. It’s so fast it’s like a flicker and I’m almost worried it will induce a seizure. Okay, that was hyperbolic, but it *is* annoying and eye-catching, feeling very frenetic in comparison to the overall feel of the WordPress admin otherwise. I’d vote for getting rid of the fade in/out. The placement and color differentiation (which is visible even with color blindness) act as ‘this is an alert’ visual cue. I would like us to add an icon or something, maybe, to bring it home, but would suggest waiting until 3.0 for that so we have time to work out proper iconography (an error message should get a different graphic than a simple confirmation message, for example).
This one seems closed, so no point in discussing it.
Alternatively plugin authors can whitelist sites themselves using the wp_oembed_add_provider() function.
Is it not self-contradictory to call *any* plugin “core”, since by definition a plugin is an extension to, rather than a part of, core?
Further, to what are such plugins “core”? I assume thewy will not be installed by default, thus they are neither part of the core code nor part of the core download package.
We are setting ourselves up for headaches and confusion if we call such plugins “core”.
I agree with this. I think the best terms are Certified and Official.
Ugh. I’m really, really not happy about eliminating the usefulness of oEmbed by making no easy way to turn it on.
I *highly* recommend that a canonical plugin be created to allow easy enabling of this. I’ll even write the plugin if necessary.
Doesn’t the change just impact oEmbed *discovery*? All whitelisted oEmbed sites would still work as expected, right? (Or am I misreading the change?)
You’re correct, but IMO, discovery is the best part of oEmbed. To be able to simply plug any sites URL in, without specific support for that site, and have it magically work… That’s very cool.
Using a whitelist system implies a lack of trust in the admin (or anybody with unfiltered_html capabilities). The admin owns the site, if he wants to post embedded code from an untrusted site, then that’s his business.
I have no problem with limiting discovery to unfiltered_html. I do have a problem with disabling discovery by default and not having any way to enable it.
The change is only to remove the option to skip the whitelist.
Due to the way oEmbed works and the trust you need to place in the sites you are embedding I strongly believe that a whitelist is the only secure way for this feature to work out of the box. The hooks are there to make it easy for a plugin to override the whitelist or add sites to the whitelist.
Yes, I know. That’s exactly the change I don’t like.
I, as an administrative user, have the ability to post unfiltered_html code, and I don’t want to use any sort of “whitelist”. Making me have a plugin to disable this part of the WordPress code is asinine.
If you don’t support oEmbed discovery, then IMO you aren’t really supporting oEmbed. You’re removing the best part of the whole thing and intentionally crippling it to just a few selected sites.
And the whole idea of making somebody install some kind of “per-site” plugin to modify the whitelist is equally silly. The thing doesn’t need a whitelist to function correctly.
You want to be safe by default, fine. Leave the option there and put a big “warning, not safe” label on it. I’m perfectly happy driving without training wheels.
The option would still be there, there’d just be no UI for it. A plugin would have to enable the discovery.
Another possible solution is to have discovery enabled for the [embed] shortcode but disabled for the URL-on-it’s-own-line. There’s a chance someone could accidentally embed something by posting it on it’s own line, but if they go to the trouble (via a UI or manually) of wrapping it in the shortcode, then we can be sure they want to embed from that site.
So now I need a plugin to eliminate the need for the shortcode? Equally silly.
I want full oEmbed. I don’t want some crippled half-assed version of it.
Great that you post a summary now, thanks.
Posted on dev blog re canonical plugins, and posted poll to weigh in on what to call them. http://wordpress.org/development/2009/12/canonical-plugins/ #wordpress
I still have my reservations about “canocial” plugins. Why there might be more than one developer and they be 100% complaint with WordPress when we go to debug code and then an author is having plugin trouble how can we be sure the plugins are not the ones causing it if we say ‘They work with wordpress 100%’ in which they don’t.
It would be opening more holes to finding problems and I am afraid that it’s going to staunch growth once we create official plugins that already exist.
Maybe a certification process instead, that we can say “This meets the security requirements of x.x.x version.” I know we do that with the new “Compatibility” box, but that is no way of saying that a plugin is secured.
Plus these types of plugins should not be full feature items. Some things are better in core, but that would be decided once the plugins that are going to be developed are known.
I found your comment highly confusing, but the purpose of canonical plugins is so that rather than having 10 plugins that all do the same thing (say announce new posts on Twitter), the authors of those plugins combined their forces and develop one common, officially supported and promoted plugin. This is much like WordPress itself — we could all develop our own blogging softwares, or we can combine forces to create a single great project. Nothing is stopping anyone from continuing to make their own version of a plugin, but by making a canonical version, it ensures it’s up to date, keeps working, receives bug fixes, etc.
A really good example of why this is needed is podPress. It has over a quarter million downloads, but the author is no longer able to maintain it and a lot of people didn’t upgrade their WordPress because podPress didn’t work with the new version. This is bad. Forking it or having someone else take over also aren’t ideal solutions as what’s stopping it from happening again?
Instead we could make podPress (or something like it) a canonical plugin where it’s officially maintained and contributed to by multiple authors. It’s a plugin that’s really needed by a lot of people (podcasters) but isn’t really something that should probably be in core (lotta users, but they make up a low % of overall users).
I’m not really sure what to think of this concept of “Canonical Plugins”.
With the concept you presented, I have the fear that we are moving towards a “two-tier plugin society”, where non-canonical plugins will be valued less than they are now. This somehow sounds like an “App Store” for plugins, though not with all of [large computer company]‘s restrictions. Questions that I see: What are the minimum technical requirements to be “canonical”? What are functionality requirements (i.e. a small plugin that filter “the_title” is most probably secure, but what’s its value)? Who decides which plugin is considered “canonical” and when? Are there any candidates right now?
I would like to view “canonical plugins” more as core functions that are not in core but in a plugin. (This, for me, would lead to the name “Core Plugins”.) That way, people could be using/activating only what they need, with the purpose of running a WP that only loads what it needs. Examples for this could be the new image editing functions. They could be in a plugin and people who absolutely want to use [fance desktop graphics program] can disable the plugin. Those plugins would/could still be maintained by core developers (or/and strongly connected developers). Another example is Akismet, which I see as a Core Plugin, as it shows the way how those plugins could be added to core. (Yet there could be an additional screen on the Plugins page with “Core Plugins”, to more emphasize that they are activily maintained.)
What you’re suggesting is pretty much exactly what we envision.
Now that I read your post again, I don’t really know anymore what triggered me to write the above entry. So, I guess, that’s a good thing
Let’s see how it turns out.
Yes, one of my suggestions was to make a new “anti-spam” plugin that’s included with core download instead of Akismet, and have Akismet be one of the services it supports as well as Typepad, etc.
It would be a good thing that some “heavy” features of WordPress would be present as “canonical” plugins to allow WordPress to be “lighter”.
Another thing that I would like to see, is the ability for plugins to have switches to turn on/off their features. This would allow users to have only the features they need turned on, and WordPress would only load these features while being executed.
How about porting some features of WordPress to these plugins to make the core engine lighter ?
Light is a function of user interface and speed. We can and have made WordPress lighter without removing functionality before.
@jane I prefer “Plugin Frameworks” over canonical, but I know many in the WP community shy away from the word framework because themers have turned it into a dirty word.
Framework is not a dirty word… A framework is what you used to build something useful. A plugin framework would be used to build other plugins, not to be used as a plugin directly.
Either way, framework only says ‘developer tool’ to most folks
Jacob Santos and 5 others are discussing. 
miqrogroove and 4 others are discussing. 
Michael Pick, 
Pulse and 13 others are discussing. 
miqrogroove 4:40 am on December 25, 2009 Permalink |
Hi Matt, consolidate profiles, that’s awesome! I’d like to see links going to the Codex user pages, as until now that was what I considered my “profile”.
Matt 4:42 am on December 25, 2009 Permalink |
WP.org profiles are far more common, and Mediawiki is being deprecated anyway, so probably won’t change the links to go to Codex instead.
miqrogroove 4:46 am on December 25, 2009 Permalink
Ah yes I didn’t mean the username links, but perhaps a link on the profile itself, perhaps in the sidebar widgets?
Matt 4:56 am on December 25, 2009 Permalink
Ah I see, that makes more sense now.
miqrogroove 5:00 am on December 25, 2009 Permalink
Merry Christmas, Matt!
Dion Hulse 6:18 am on December 25, 2009 Permalink |
Any chance of the profile showing trac comments? Would be nice for some end users, May overload things for common trac users though.
(also, Whats with the tabbing order of this form, Name -> Email -> Email Notify updates -> Website?)
Dion Hulse 6:20 am on December 25, 2009 Permalink |
Also, Maybe being on the Profiles page should mark a tab as the selected one? Right now, it looks like the Download tab is the selected one.. (I’m talking the Global nav menu)
PeteMall 6:48 am on December 25, 2009 Permalink |
Ho Ho Ho… Merry Christmas!
miqrogroove 5:57 pm on December 25, 2009 Permalink |
I’m messing around with Gravatar as well. After updating my settings, the image URL still has a header that says Last-Modified: Wed, 03 Sep 2008 16:16:39 GMT. I only noticed it because the Reload button in Chrome doesn’t clear out the old version. The Expires interval is only 5 minutes. I haven’t tried an If-Modified-Since to see what the result is, but you get the idea?
nacin 2:22 am on December 27, 2009 Permalink |
Matt,
Just a heads up, I think this broke the ability to preview a comment on Trac. If you could look into that, that would be quite helpful.
Jacob Santos 10:11 pm on December 31, 2009 Permalink |
I’ve always found the profile to not pick up all of my contributions, which makes it worthless for something that I could show someone. I could show them and be like, “Well, this is a few of the things I’ve done, but it doesn’t have the many many more tickets and patches I’ve done over the past 2 years.”
I’ll probably just have an overview of the stuff I’ve done for WordPress on my own site linking to tickets, if I wanted to do this at all. The problem is that the “official” will conflict with my version and might look like I’m fibbing people of the things I have done in the past working on WordPress.
I’d rather the feature to remove this completely until which time the feature is improved.
Denis de Bernardy 9:56 pm on January 5, 2010 Permalink |
One small annoyance crept into trac: comments no longer have the handy pound with a link to particular comments.
Other small annoyance, in P2 this time: whatever plugin you’re using in P2 for new comment/new post notifications looks all weird when replying to a comment.
Denis de Bernardy 5:56 pm on January 8, 2010 Permalink |
@Matt: can anyone look into re-introducing the # with a permalink to individual comments in trac?
Matt 8:23 pm on January 8, 2010 Permalink
When you hover over the line above a comment a little paragraph tag appears, which is the permalink.
Denis de Bernardy 1:27 am on January 9, 2010 Permalink |
Ah, there it is. I was no longer seeing it for some reason. Thanks!