Add-on Incubation Proposal

<<

fligtar

Add-ons Product Manager

Posts: 132

Joined: Tue Sep 22, 2009 8:00 am

Location: SF Bay Area

Post Mon Mar 22, 2010 3:54 pm

Add-on Incubation Proposal

This is a proposal for changing the way unreviewed add-ons are presented and hosted on addons.mozilla.org. For background information, please read this blog post announcing the proposal, and then read the full proposal here.

A summary of its points:
1. Unapproved add-ons will not be shown in the add-ons gallery.
Users can be sure that every add-on they come across browsing around the website has been tested and approved by Mozilla’s editors. This would mainly affect browsing through categories, search results, and collections.

2. Users who are linked directly to an unapproved add-on by its developer can install the add-on.
After submission, developers can send the link to their unapproved add-on to anyone they’d like to try it out. This can be on their blog, website, email, instant message, etc. However, these direct links will be disjointed from the normal add-ons gallery and make it clear that this add-on has not been reviewed, as well as offer the user the option to leave and go to the gallery with only reviewed add-ons.

3. There will be a maximum amount of time an unapproved add-on can be hosted on the site.
There are 7,500 add-ons currently in the sandbox that have never been reviewed by a human because the developers have never nominated them to show up in the “public” site. Some of these add-ons have been there for years, and are usually the source of the problems we do have.

To address this, we think a critical part of fixing the review process is setting the expectation that all add-ons submitted to the site must undergo review and using a time limit to enforce that.

Under this proposal, an add-on’s submission to addons.mozilla.org will begin its 30-day incubation period. As noted above, the add-on will not appear in the gallery but can be accessed through a direct link from the developer during this period.

The add-on must apply to the gallery sometime during this incubation period, whether on day 1 or day 30. Reminder emails will be sent at certain intervals to remind the developer of this timeframe, but if the add-on has not applied to the gallery during the 30-day window, its listing will expire and no longer be available on the website, even with a direct link.

If the developer has allowed an add-on to expire, he or she may still submit a gallery application after the 30-day window, but the add-on will not be re-activated for download until it has been approved.


I'd love to hear feedback from add-on users and developers on these ideas.
<<

archaeopteryx

AMO Editor

Posts: 1036

Joined: Fri Sep 25, 2009 1:38 pm

Location: Dresden, Germany

Post Mon Mar 22, 2010 4:50 pm

Re: Add-on Incubation Proposal

Please provide reasons for the changes.

fligtar wrote:Limit the amount of time an add-on can be hosted on a trusted Mozilla site without being reviewed by a human
How does this help anybody? Malicious files will be used in the first 30 days.
fligtar wrote:Prevent exposure of untested add-ons to consumers casually browsing the gallery
You can raise the awareness by changing the wording to something more sensible.
fligtar wrote:Continue to provide hosting for newly created add-ons so that the burden of setting up a website and installation process is not a barrier to add-on development
Needs info how people interested in fresh meat can explore it.

fligtar wrote:Upon submission, a new add-on will be in a 30-day incubation period where:
I know many useful extensions which have never been reviewed or took a long very time to get mature, so I oppose this for a few reasons:
    Will increase the burden on editors to review files which aren't yet ready for public because of scarce resources of the developer
    What happens if it is nominated for public, but doesn't get reviewed
    Listing it longer won't cause more harm than currently

fligtar wrote:it will not show up in any browse, search, collection, or other public listings

Will there be a search to find something independent if it is in the "gallery" or in the incubator?

fligtar wrote:it can only be accessed directly through its details page, as linked to from an external source, such as the creator's blog or an email/conversation with a friend invited to try the add-on

So the developer should personally lobby to spread his or her extension and people interested in this who don't know him won't participate? You are raising the bar foo feedback during development very much.

fligtar wrote:the URL will start with /incubator
Hopefully localizable. If the file is not directly installable by clicking a direct link from an external page (i.e. by AMO cookie setting roadblocks), you should concentrate more on the details page

fligtar wrote:allow us to disable crawlers
Crawling should be enabled, use better wording and if you want, registration for installing.

fligtar wrote:the automatic update service will function for all add-ons hosted on AMO, including unreviewed add-ons in incubation
So if I review the code on my own, I can get a bad surprise later? Unreviewed add-ons should be opt-in, not forced or opt-out.

fligtar wrote:New versions [...] will not be available for public download until they have been tested.
I feel not comfortable about that. If a person who isn't an editor submits problems before the potential review, this is a gain.
<<

cfinke

Posts: 1

Joined: Mon Mar 22, 2010 5:17 pm

Post Mon Mar 22, 2010 5:33 pm

Re: Add-on Incubation Proposal

As a developer, I have no problem with #1 or #2. The only issue I see with #3 is that, in my experience, getting someone to leave a review of a sandboxed add-on can be like pulling teeth, even if they enjoy using it. Case in point, Twitter Filter: https://addons.mozilla.org/en-US/firefox/addon/55159/ Multiple people have told me that they appreciate it, and more than a few have told me that they would rate it and write a review, but it remains unrated and unreviewed. Because it is three months old, it wouldn't be visible under the new policy. (Once an add-on is made dormant, will existing uses still be able to leave reviews?) I could always apply for public status, but without any reviews, it would be denied.
<<

archaeopteryx

AMO Editor

Posts: 1036

Joined: Fri Sep 25, 2009 1:38 pm

Location: Dresden, Germany

Post Mon Mar 22, 2010 5:43 pm

Re: Add-on Incubation Proposal

Reviews aren't needed any more for going public (for nearly a year, if I remember correct). So feel free to nominate. Editors can approve it without reviews from normal users.
<<

fligtar

Add-ons Product Manager

Posts: 132

Joined: Tue Sep 22, 2009 8:00 am

Location: SF Bay Area

Post Mon Mar 22, 2010 7:31 pm

Re: Add-on Incubation Proposal

Archaeopteryx wrote:
fligtar wrote:Limit the amount of time an add-on can be hosted on a trusted Mozilla site without being reviewed by a human
How does this help anybody? Malicious files will be used in the first 30 days.

With the current state of things, a malicious file will be on the site until someone reviews it. If the author never asks us to review it, we don't find it until someone figures it out and reports it to us. In a recent example, an add-on was downloaded for 5 months before we were told about the trojan and could take action.

In the proposed system, the maximum amount of time any one file would go unreviewed is 1 month, greatly reducing the number of people who can have a chance to download it. It's certainly not perfect, but it is a big improvement.

Archaeopteryx wrote:
fligtar wrote:Prevent exposure of untested add-ons to consumers casually browsing the gallery
You can raise the awareness by changing the wording to something more sensible.

Yes, we plan on doing that, as part of this layered approach.

Archaeopteryx wrote:
fligtar wrote:Upon submission, a new add-on will be in a 30-day incubation period where:
I know many useful extensions which have never been reviewed or took a long very time to get mature, so I oppose this for a few reasons:
    Will increase the burden on editors to review files which aren't yet ready for public because of scarce resources of the developer
    What happens if it is nominated for public, but doesn't get reviewed
    Listing it longer won't cause more harm than currently

Sure, not every add-on will be full-featured right out of the gate, but the functionality that is offered upon upload should work and be free of security problems, which is what the review will be checking. If an add-on is one of the edge cases that takes a long time to review (likely due to admin review of binary components or a similar situation) then it will remain in the incubator until it is reviewed.

Archaeopteryx wrote:
fligtar wrote:it will not show up in any browse, search, collection, or other public listings

Will there be a search to find something independent if it is in the "gallery" or in the incubator?

No, add-ons that have not been reviewed will never be surfaced on any page except their details page for people that are given the link. We don't want users to accidentally stumble upon them when browsing Mozilla sites.

Archaeopteryx wrote:
fligtar wrote:it can only be accessed directly through its details page, as linked to from an external source, such as the creator's blog or an email/conversation with a friend invited to try the add-on

So the developer should personally lobby to spread his or her extension and people interested in this who don't know him won't participate? You are raising the bar foo feedback during development very much.

The developer doesn't need to personally lobby -- if he or she wants to share it with their friends early, the direct link can be used. Otherwise, after it is approved for the gallery, everyone will be able to find it.

Archaeopteryx wrote:
fligtar wrote:allow us to disable crawlers
Crawling should be enabled, use better wording and if you want, registration for installing.

One of the main points of this proposal is preventing people from randomly stumbling upon these add-ons. We want to disable crawlers so that a search for "tab add-ons" doesn't return anything that hasn't been reviewed. I think there's a difference between annoyance and warning here -- making people register to download an add-on is annoying and unnecessary when a warning will sufficiently make the point we're trying to make.

Archaeopteryx wrote:
fligtar wrote:the automatic update service will function for all add-ons hosted on AMO, including unreviewed add-ons in incubation
So if I review the code on my own, I can get a bad surprise later? Unreviewed add-ons should be opt-in, not forced or opt-out.

If you review the code on your own, presumably you would know that if an update is offered, the code has changed and you should review that code before accepting the update. As add-on updates don't currently automatically install, these updates are opt-in.

Archaeopteryx wrote:
fligtar wrote:New versions [...] will not be available for public download until they have been tested.
I feel not comfortable about that. If a person who isn't an editor submits problems before the potential review, this is a gain.

I'm not sure what you mean here -- can you explain?
<<

cdolivei

AMO Editor

Posts: 1

Joined: Mon Mar 22, 2010 9:48 pm

Post Mon Mar 22, 2010 10:16 pm

Re: Add-on Incubation Proposal

I think we should just pop-up a image of Admiral Ackbar when a user installs a sandbox add-on.

Seriously though, while I am in favour of change, I think the new system could be simpler. There doesn't seem to be any advantage of nominating later rather than sooner. 30 days isn't enough time for most developers to attain a large enough user base to find the subtle bugs (I assume that's why the 30 day window exists?). One of my extensions took months of word-of-mouth, some advertising between friends, and general user curiosity before I was ok with nominating it. An additional 7 days doesn't make sense to me. I guess to fix and resubmit any problems during the review stage, but once again all the more reason to submit sooner. Maybe the extensions should be auto-submitted by default.

But I do agree that we should limit some of the noise in search results. Some extensions, IIRC, were denied in the past because editors couldn't test (external software requirement, worked with a small ISP, etc). I take it these will have to be self-hosted? (will self-hosted even show in the add-on gallery?)
<<

archaeopteryx

AMO Editor

Posts: 1036

Joined: Fri Sep 25, 2009 1:38 pm

Location: Dresden, Germany

Post Tue Mar 23, 2010 2:14 am

Re: Add-on Incubation Proposal

Justin Scott (fligtar) wrote:
Archaeopteryx wrote:
fligtar wrote:New versions [...] will not be available for public download until they have been tested.
I feel not comfortable about that. If a person who isn't an editor submits problems before the potential review, this is a gain.

I'm not sure what you mean here -- can you explain?


If a person who is no editor but dives into investigating new downloads (like applicants for AMO editors with not enough experience who should train), he can't do this anymore because he can't find the files. What about changing the install to a download button (would only work for Firefox and SeaMonkey)?
<<

titoo

Posts: 97

Joined: Fri Oct 23, 2009 5:11 pm

Location: MVD, Uruguay

Post Tue Mar 23, 2010 3:01 am

Re: Add-on Incubation Proposal


my english su*//s but you will get the point
I say "You" many times but I'm not referring to anyone in particular just to the situation.

Hi,
IMHO, the 30-day incubation will convert the problem into worse.

You are missing to many points
- The most important: the 'malicious' developer can submit a different add-on every day. So he will get malicious code cycling every 30 days but for ever. . And the users will get updates of these?

- Note less important: some developers do this just for fun, or because is enjoyable. And these types of things are done in free time, or school time... free time is free time and sometimes we don't have free time or we have but we are doing others things.
By shorting the exposure, you will put unnecessary pression of time over developers that are just constructing a better browsing experience.
If I feel the presion to develop with a deadline, I will probbaly feel stressed and I'll ending doing some other thing not related to the develop of add-ons. If I need to work in an add-on in a time when I'm not motivated to work on that add-on, I will probably quit, because it will not be enjoyable. Really.

- The site should be fully browsable, even these extension never nominated. How can I install "JavaScript Command" if I can't found it? a checkbox or 20 checkbox? "I'm superman, let me install and browse whatever that I want"
Review and emails from users motivate developers to do better add-ons because users inspire developers, if users can't find these add-ons then you will get less developers and less happy users trying to find that add-on that is hidden but do exactly that he was looking for.
IMO this should be managed in another way.
Also, and very important, some great developers will get frustrated by this "feature"? and will not submit their add-on to AMO, and that is a huge loss, because it will be more difficult for users found reallly great and useful add-ons, I can count some right know....

Archaeopteryx wrote:
fligtar wrote:New versions [...] will not be available for public download until they have been tested.

I feel not comfortable about that. If a person who isn't an editor submits problems before the potential review, this is a gain.

I'm not sure what you mean here -- can you explain?

He is trying to say that updates on add-ons pending approval (these not tested) may contain errors that users can found and report until these hit the review and update service.
<<

wladimir-palant

Posts: 2

Joined: Tue Mar 23, 2010 3:27 am

Post Tue Mar 23, 2010 3:51 am

Re: Add-on Incubation Proposal

I don't have a big problem with the first two points of the proposal either. However, I have my doubts about the time limitation.

First of all, not having unreviewed add-ons show up in search results should already be a sufficient measure to deal with malicious add-ons. The main problem with those isn't that they are hosted on AMO - if somebody can link to AMO for this add-on they can just as well host that malicious add-on themselves. When unexperienced users can stumble upon this add-on by doing a regular search - that's a problem. As mentioned above, a malicious author can change add-on ID every few weeks and reupload, this way he will be able to link to an add-on on AMO continuously.

So the time limitation can only be considered as a way to put some pressure on add-on developers to get their add-ons reviewed rather than keeping them in an unreviewed state indefinitely. Here again, removing this add-on from the search results creates a strong incentive to host an unreviewed add-on AMO, a time limit might be unnecessary. On the other hand, there are add-ons like my Adblock Plus Watcher - I don't want it to be public, this will make many people install it simply because it has something to do with Adblock Plus, even though it is more or less a debugging extension. I can host it externally, not a big deal, but it would still be a pity. And some people might want to keep an add-on on AMO on a temporary basis until they are confident enough to release it to public (which might take significantly longer than a month).

What will certainly cause bad blood is the 7 days extension if an add-on was rejected. Sure, you expect that people will nominate add-ons for public sooner rather than later - but you will be disappointed. You will see people ranting that review took too long and they don't have enough time to fix issues now. To prevent this you should count the extension from the date when the review is finished, this will make it independent of the length of the review. And you should probably come up with something longer than 7 days, I would say that it should be 14 days at least. Many add-on developers are hobbyists, working on the add-ons in their spare time and without the right skills. Coming up with anything in a short time is a problem for them, even if you point out exactly what they need to do. I've noticed this too well when reporting security issues. Also, see those extensions where a compatible version comes out months after a Firefox release, despite the changes being trivial.
<<

mcdavis

Posts: 6

Joined: Thu Jan 28, 2010 12:56 am

Post Tue Mar 23, 2010 4:27 am

Re: Add-on Incubation Proposal

Thanks for explaining this in detail.

About a time limit for experimental add-ons: is there any data on what percentage of existing add-ons went from first appearance on AMO to first public release in 30 days? (The underlying question being, how many of them were *able* to do this?)

How should a new add-on go about beta testing, prior to first approval? 30 days doesn't seem like enough time to go through several alpha and beta releases and evolve into a niche in the ecosystem. If anything, a new add-on (not previously submitted and approved) needs more bake time with beta users than one already approved and released. From this perspective, a long incubation period is not a minus but a plus. If there is a time limit, maybe it should be something longer, like a year.

I take it as given that add-ons need beta cycles and a gradually-increasing contact with live users over time, for the same reasons Firefox does. Are you just saying it's more important for AMO to shield users from beta add-ons than to publicize new and still-beta add-ons? And that however beta testing is done, add-ons need to show up bomb-proof and ready to roll before they show up in any AMO listing other than the direct-link page for the add-on?

Another way to limit risky exposure from unreviewed add-ons would be for them to automatically self-request a review, separate from a request for public approval, just by virtue of having been experimental for a certain period of time. Maybe not so great for reviewer workload, but an automatic review after 30 days would produce the same exposure as an add-on which is automatically pulled after 30 days without review. (Something tells me this has already been considered...)

About updates for unreviewed add-ons -- speaking for myself as an end user, I don't want updates for unreviewed add-ons. If I installed an experimental add-on, it's because I downloaded and inspected the code for the specific version installed (or sometimes because the author is trusted). An update would put unreviewed code in my chrome and that's obviously not good. Accidents happen. Maybe I clicked "Install Updates" instead of "Skip" because it's a habit, or accidentally clicked "Install Updates" instead of just updating one single one I meant to update, or maybe I didn't see the experimental add-on in the list of available updates.

I love the idea of big stern warnings on unreviewed add-ons. It's great that they would be hard for the average user to find and scary to install if they are found.
<<

wladimir-palant

Posts: 2

Joined: Tue Mar 23, 2010 3:27 am

Post Tue Mar 23, 2010 4:53 am

Re: Add-on Incubation Proposal

I checked the timeline for my JavaScript Deobfuscator add-on. I uploaded version 1.0 to sandbox, wrote about it in my blog (which luckily has enough readers), hit reality (user interface too primitive to be useful), reworked user interface, fixed some bugs. Finally nominated version 1.5.1 for public and it got accepted - 32 days after initial upload. I would consider that pretty fast turn-around time, partly because I already have significant add-on development experience, partly because I managed to get some useful feedback soon, partly because I could invest some time into that add-on (which isn't always the case). Point being: limiting sandbox to 30 days is equivalent to requiring add-on authors to host the add-on outside AMO during its experimental stages and only upload to AMO once they gathered enough user feedback and are confident that it is ready for prime time. Whether that's a good thing is questionable.
<<

idevfh

Posts: 128

Joined: Thu Oct 29, 2009 8:22 am

Post Tue Mar 23, 2010 7:04 am

Re: Add-on Incubation Proposal

I am grateful to Mozilla to allow my add-ons to be hosted and of course all the benefits from the hosting I received.

But, the 30-day incubation period is just way to short for an experimental add-on to be tested out by users and to be able to get feedback.
Many of us developers have other jobs that prohibit us from full-time devotion to an add-on. Mozilla should consider most of the developers supplying the add-on extension to Firefox user, don't have funding, or the convenience to watch the add-ons reviewing activity all day long to modify any corrections that are pointed out by a reviewer's comment. At least not until the later hours of the day or night.

As far as protection to the users the percentage of malicious developers taking the time to promote bad add-ons within Firefox isn't that high, "Is It" ?

A typical experimental add-on page does suggest the following:
Let me install this experimental add-on. What's this?
Add to Firefox (Windows)
experimental


Regards,
IDEVFH
<<

customfirefoxlady

Posts: 53

Joined: Wed Oct 28, 2009 5:22 am

Post Tue Mar 23, 2010 7:32 am

Re: Add-on Incubation Proposal

I agree that we need to make unreviewed add-ons less visible to the average user.

I think that there are two distinct issues as to reviews:
1. Free-from-malware and 'no surprises'
2. Acceptable quality (free from excessive bugginess, etc.)

Ideally review for #1 should occur as soon as is realistically possible. #2 can take considerable time to develop, especially for new hobbyist add-on developers. We shouldn't want to promote not-ready-for-prime-time but I've-got-to-beat-the-deadline add-ons.

Therefore I think a two-tier system should be considered. A review for #1 in a very limited time period, followed by an optional (developer's discretion) beta period (or whatever you want to call it) where users can opt-in (after being warned what beta-quality means) to view and test without the need for a direct link. Then a review for #2 which can take into account any user ratings/reviews. Perhaps that is too much workload on the editors, but even the original proposal would add to the current workload, and review #2 should be do-able by people with less code-knowledge.

Speaking of beta, it's not totally clear from the proposal, how it would affect the recently added beta channel for updates. Would those updates be reviewed before being made available?

Wladimir Palant wrote:... Many add-on developers are hobbyists, working on the add-ons in their spare time and without the right skills. ...
LOL, but yes. And they can produce useful and highly-rated add-ons and develop those 'right skills', but it takes time.
Wladimir Palant wrote:...Point being: limiting sandbox to 30 days is equivalent to requiring add-on authors to host the add-on outside AMO during its experimental stages and only upload to AMO once they gathered enough user feedback and are confident that it is ready for prime time. Whether that's a good thing is questionable.
Exactly. And less exposure means bugs take longer to find and fix.
<<

nils-maier

AMO Editor

Posts: 3

Joined: Tue Mar 23, 2010 7:57 am

Post Tue Mar 23, 2010 8:12 am

Re: Add-on Incubation Proposal

I don't really like #1, no AMO exposure at all.
While I fully agree that unreviewed addons should not show up in the gallery or searches for "regular" users, I'd like a way for users to enable listings again.
I therefore propose to include an opt-in option in the account preferences that enables listings again (with a scary warning, of course).
So, to browse one needs an account, being logged in to that account, opt-in, and still follow the 2-clicks installation process.
(Direct linking, as proposed, should be still possible)

This way experimental addons will get some more exposure, which will help finding bugs faster and implementing refinements while still protecting regular AMO visitors from the evils of unreviewed addons.
I received a fair share of feedback for my experimental addons that were not announced anywhere expect AMO, and that really helped me fixing bugs and other issues before the addons became public. Would be a pity if developers that do not have a popular blog don't get any feedback at all.
<<

nik4name

Posts: 4

Joined: Fri Feb 05, 2010 4:53 am

Post Tue Mar 23, 2010 1:39 pm

Re: Add-on Incubation Proposal

this is going to be a choice between
a) keeping site enjoyable and friendly for developers and early users
and
b) not allowing new users to do something they don't understand, and preventing any mention of malware and AMO together
your proposal takes too much care for the second, and very little for the first:

1) it's wrong to hide new addons and new versions from users who doesn't want that "extra care for their safety",
and don't leave any way back, like registering as tester or something
removing beta channel will be wrong as well

2)
Justin Scott (fligtar) wrote:After submission, developers can send the link to their unapproved add-on to anyone they’d like to try it out.
the testers and developers don't know each other. now mainly AMO helps to find testers

3)reviewing all addons is good, but
custom.firefox.lady wrote: think that there are two distinct issues as to reviews:
1. Free-from-malware and 'no surprises'
2. Acceptable quality (free from excessive bugginess, etc.)

it's better to have addon which does what one needs, but doesn't follow guidelines (and may break something), than none at all

maybe making addons-incubator.mozilla.org or test-addons.mozilla.org as a place for development and testing
and leaving addons.mozilla.org for only finished, high quality addons will be better solution for all?
<<

sdwilsh

AMO Senior Editor

Posts: 5

Joined: Mon Mar 22, 2010 8:06 pm

Post Tue Mar 23, 2010 1:55 pm

Re: Add-on Incubation Proposal

nik4name wrote:it's better to have addon which does what one needs, but doesn't follow guidelines (and may break something), than none at all


I can't say I agree with you. Users aren't going to know why something broke, just that it broke. It may solve one need, but then create a problem so you haven't really gained anything.
<<

webgapps

Posts: 100

Joined: Sun Jan 17, 2010 5:17 pm

Post Wed Mar 24, 2010 2:12 am

Re: Add-on Incubation Proposal

I understand the objectives of the proposal and I think they are reasonable, although I think the expiration period could result in rushed submissions, lots of extra work for the editors and frustrated developers, who will eventually migrate to self-hosted add-ons.

And what about the self-hosted add-ons? Any plans to limit their exposure? I think they should be the first target. I recently subscribed to the new add-ons feed and I get a lot of self-hosted toolbar extensions an theme series with just subtle changes. Very annoying. In my opinion, self-hosted add-ons should be kept completely separate from the reviewed add-ons gallery.
<<

luckyrat

Posts: 9

Joined: Wed Mar 24, 2010 2:56 pm

Post Wed Mar 24, 2010 4:04 pm

Re: Add-on Incubation Proposal

Hi all,

I'm a user of add-ons and a developer of an add-on for users. I fully support the goal of hiding "beta / untested / experimental" from the end user (and have thought as much for some time *slaps self on wrist for not contributing sooner*). Most users should never need to see these add-ons so hiding them from search results, browse lists and robots are great ideas. As is enforcing automatic updates on non-reviewed add-ons - that's one of the reasons my add-on (KeeFox) is not currently in an experimental state on AMO (users would expect it to update automatically, even more so than they probably already do from the privately hosted location!)

I also agree with a lot of the posts here and on the blog:

1) The time limit has many disadvantages and will eventually harm end users by discouraging developers (particularly new hobbyists), hence lowering the variety and quality of add-ons available on AMO. It might also lead to fragmentation and confusion, with users bookmarking multiple add-on sites because the add-ons they want are not available on AMO.

2) A way for advanced users to find un-reviewed add-ons is important in order to encourage advanced users to provide feedback on experimental add-ons.

Some suggestions:

1) Abandon the time limit idea entirely.
2) Set very different time limit criteria - e.g. "no new version uploaded for 6 months" leads to a de-listing?
3) Put even more dire warning messages all over the page of any add-on which has been unable to meet your time limits - e.g. "this add-on has been in an un-reviewed state for a very long time so treat with extra caution".
4) Allow users to opt-in to seeing un-reviewed add-ons (whether it's a one-off advanced search option, permanent user profile setting or something in between).
5) Maybe you could implement 3 and 4 but hide those extremely old add-ons even from the advanced search results?
6) Concerning brand trust, I would support making the non-reviewed add-ons pages VERY different from the main add-on site - users have to know immediately that they are not in the usual trusted location, a different coloured button isn't sufficient by itself; that applies whether they access the page through a direct link or advanced search on the main AMO site. (Are there any mock-ups yet or is it too early?)

I strongly encourage you to abandon the automatic time limit but if it must remain, please at least choose time-scales that more closely match those that the many add-on hobbyists can work to. I put a lot of time into KeeFox but I still think it's unlikely that I'd be able to respond to a failed submission request within 7 days. Or must I now plan my holidays around this arbitrary time window?

I hope you can take these suggestions into account but regardless, I'll reiterate that I do support the overall objectives and I look forward to seeing (some of) these proposals implemented!

Thanks,
Chris
<<

fligtar

Add-ons Product Manager

Posts: 132

Joined: Tue Sep 22, 2009 8:00 am

Location: SF Bay Area

Post Wed Mar 24, 2010 4:50 pm

Re: Add-on Incubation Proposal

nik4name wrote:this is going to be a choice between
a) keeping site enjoyable and friendly for developers and early users
and
b) not allowing new users to do something they don't understand, and preventing any mention of malware and AMO together
your proposal takes too much care for the second, and very little for the first:


One of the challenges that AMO faces in many things is balancing between the needs of developers and consumers. In this case, I do think we're leaning towards consumers, and I think that's the right thing to do here. An add-ons site with all consumers and few developers is obviously something we really want to avoid, but so is an add-ons site with all developers and no consumers. AMO is primarily an add-on distribution and discovery service, not so much project hosting. We've never tried to be project hosting for fledgling add-ons, although there certainly are projects that focus on that, such as MozDev.

I think this proposal is AMO making itself more clearly a gallery for cool, useful add-ons and less for unfinished projects or buggy experiments, while still giving developers a place to distribute those works-in-progress for a little while.

In reading through feedback, the majority of folks are fine with not showing unreviewed add-ons in the gallery and instead letting those add-ons be accessed through a direct link. The contentious issue seems to be the 30-day incubation period, and people seem to be divided between not wanting a time limit at all and thinking 30 days is too short.

I ran some stats on the 139 new add-ons that were in the nomination queue awaiting review yesterday. 20% of those add-ons were nominated in the first hour they were submitted to the site, 32% in the first day, and 60% in the first month. So, this new proposal's incubation time period would affect the 40% of people that are currently taking longer than a month after submission to nominate their add-on for review. I think we can consider bumping the 30 days up to 60 days, but that would only include a few more add-ons from the 40%. Does 60 days sound like a more reasonable number?

Arguing that people could still submit malicious add-ons during the 30 day period really only helps the case for removing unreviewed add-ons entirely, as some want to do. Providing the incubation period was a compromise in hopes of improving the current situation, in which malicious add-ons could live for much longer than 30 days undetected as long as they don't ask for review.
<<

archaeopteryx

AMO Editor

Posts: 1036

Joined: Fri Sep 25, 2009 1:38 pm

Location: Dresden, Germany

Post Thu Mar 25, 2010 3:21 am

Re: Add-on Incubation Proposal

[quote="Justin Scott (fligtar)"]In reading through feedback, the majority of folks are fine with not showing unreviewed add-ons in the gallery and instead letting those add-ons be accessed through a direct link.[/url]
I don't see this in the comments. Most people agree that a normal user/visitor shouldn't find them. Hiding by default and at least a possibility to enable them would be fine. And as mentioned, AMO also helps to find testers for the young add-ons.

If you tell the developers that they should better use mozdev, why should they still host it on AMO? Most of them don't monetize it, so no gain for additional work.
<<

jorge-villalobos

AMO Administrator

Posts: 2989

Joined: Tue Sep 29, 2009 7:30 pm

Location: San José, Costa Rica

Post Thu Mar 25, 2010 8:29 am

Re: Add-on Incubation Proposal

Archaeopteryx wrote:If you tell the developers that they should better use mozdev, why should they still host it on AMO? Most of them don't monetize it, so no gain for additional work.


Infinitely better exposure, free code reviews, user reviews, contributions... the list goes on. I don't think it's such a bad idea to use Mozdev for projects that are just taking off. It has a more developer-oriented community that can help identify bugs and bring projects to a more stable footing before trying to nominate on AMO.

As Justin said before, this is not an easy decision we're making. It's a compromise between what gives developers the most exposure and what is considered safe, which would be to have no sandbox at all.
<<

archaeopteryx

AMO Editor

Posts: 1036

Joined: Fri Sep 25, 2009 1:38 pm

Location: Dresden, Germany

Post Thu Mar 25, 2010 8:39 am

Re: Add-on Incubation Proposal

Compared to a time limit, I would prefer the drop of sandbox version (and people hosting it on mozdev/theirs owns/somewhere else). Every disappearing file or even add-on (especially with an open source license) hurts people interested in building extensions.
<<

sdwilsh

AMO Senior Editor

Posts: 5

Joined: Mon Mar 22, 2010 8:06 pm

Post Thu Mar 25, 2010 8:42 am

Re: Add-on Incubation Proposal

Archaeopteryx wrote:Compared to a time limit, I would prefer the drop of sandbox version (and people hosting it on mozdev/theirs owns/somewhere else). Every disappearing file or even add-on (especially with an open source license) hurts people interested in building extensions.

How is this better than the proposal? At least with the proposal people can link to their add-ons on AMO which people, in theory, trust more than some random site.
<<

archaeopteryx

AMO Editor

Posts: 1036

Joined: Fri Sep 25, 2009 1:38 pm

Location: Dresden, Germany

Post Thu Mar 25, 2010 9:09 am

Re: Add-on Incubation Proposal

That's the problem, they shouldn't trust these files more. And links will be dead after the period if the developer didn't manage to get it ready for public.
<<

nik4name

Posts: 4

Joined: Fri Feb 05, 2010 4:53 am

Post Thu Mar 25, 2010 1:54 pm

Re: Add-on Incubation Proposal

Shawn Wilsher (sdwilsh) wrote:I can't say I agree with you. Users aren't going to know why something broke, just that it broke. It may solve one need, but then create a problem so you haven't really gained anything.

it may break but usually it doesn't, and users should be free to choose themselves.
Passing review doesn't mean that addon will work with others, and only way to know that is letting people to try.

What is really bad about this proposal, is the assumption, that people don't understand what are they doing, and only way to keep them safe is to hide all untested things, and put warnings all over the place.

There are users who actually want to try new things (like untested addons, new versions, ff nightlies) as soon as possible. and having to go to new site with unfriendly interface is much more annoying than registering as a tester.

Having a page with listing of untested addons only, wouldn't hurt anybody, will make things better for this users and help to make sure that addon's are really working before going public.

Тhere are two things unclear in the proposal.
will add-on developers be able to put direct links to their new untested addon on other addons page?
and will beta channel remain?
Next

Return to Announcements and Roadmap

Who is online

Users browsing this forum: No registered users

Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
CA Gen2 style designed by Vjacheslav Trushkin.