19

One of the criticisms that the moderators / staff of the StackExchange community seem to have brought up is the lack of turnaround between commitment sign ups and beta users.

Specifically these comments from @JNAt:

enter image description here

And here:

enter image description here

This thread captures a look into that issue, and hopefully enables clarification from the moderator community about the behavior.

Expectations

First, the community size that is required to leave the commitment phase and enter beta is 200 committers. On top of that, 100 committers have to have had at least 200+ reputation on other StackExchange sites.

Anything beyond that is "extra" from the metrics provided to the community stewards.

Commitments

The commitment phase of Substrate gained 642 total commitments, of which 120+ users had over 200 reputation.

There is then there is a long tail of users with 50 or less reputation, who are categorized as new users who are interested in this topic.

Among those that made referrals in the commitment phase, one person stands out in particular:

enter image description here

If we do some basic look into the referrals this user made, most of them seem to be brand new StackExchange users. Here is just an example page:

enter image description here

It seems to me this is unfortunately an example of someone reaching out to their community to "pump up" the commitment numbers, where these individuals have not really ended up engaging our beta community.

Obviously this is NOT representative of our overall community, and we did NOT need these users to pass your bar, however, it seems now these users are being used AGAINST us when trying to qualify the quality of our proposal.

Delay

The conversion from our StackExchange commitment to our Beta took around 3 months. In that time, we went through holidays, a new year, and, if you keep up with the news, the world around us has changed quite a bit.

In general, the moderators and staff are looking for "an explosion of activity" from these committers, but have also contributed to killing the flame by having such a delayed process.

We are sorry we proposed our StackExchange community near Christmas.

Beta

Now we are in Beta, and only a private beta, we can actually get a feeling of how many users are in our community.

JNat points out that "only 22% showed up" yet, our beta now has over 900 users, which is a 40% increase over our ENTIRE commitment community.

Moving Forward

The Substrate community does not want to encourage or need help from "pumping up numbers" by community who are not actually interested with engaging our StackExchange community.

As you can understand, we cannot control the actions of others, but similarly, we should NOT be punished by those actions either.

I suggest you remove all of the referrals from Ray, and future measurements of the success of our site be qualified against the REAL numbers that our community brings to this site.

It seems completely unjustified at this point to qualify the lack of success of this proposal to this metric, and if the staff or moderators disagree, I would like to hold that conversation of why here.

Otherwise, I would appreciate confirmation that we are satisfying this criteria, and doing so in a completely natural way.

1 Answer 1

7

If you compare the ratios of followed, committed and signed up with other similar Stack Exchange sites that successfully went through their beta phase:

https://area51.stackexchange.com/proposals/89704/ethereum

  • 293 users committed
  • 161 users followed

https://area51.stackexchange.com/proposals/30763/bitcoin

  • 253 users committed
  • 150 users followed

https://area51.stackexchange.com/proposals/125174/cardano

  • 514 users committed
  • 343 users followed

https://area51.stackexchange.com/proposals/120123/tezos

  • 256 users committed
  • 132 users followed

https://area51.stackexchange.com/proposals/116927/eosio

  • 543 users committed
  • 312 users followed

https://area51.stackexchange.com/proposals/98617/monero

  • 277 users committed
  • 125 users followed

The Substrate site has an unusually high committed rate relative to the followed rate which makes the % of users who signed up afterwards look small:

  • 642 users committed (23.8%)
  • 167 users followed (49.1%)

Based on the ratios for other sites above, the substrate site should've had around around 350 committed users - which would make the sign up percentage 43.5%.

And as mentioned, It's likely due to someone pumping up the committed numbers with low-rep users at late stages - where what we needed was high-rep users - and those low-rep users never signed up afterwards (might have even been fake accounts).

Ideally someone would be able to comb through the committed user list and remove users who have had no-activity on their accounts since they committed to the site or otherwise any other accounts that look suspicious or inactive.

1
  • 3
    Well regardless of initial committed users, it seems like there's a constant stream of new faces using the site which is encouraging. It's not just repeat people coming back for more. The one puzzle I have is why the site says that 0% have completed their commitment. This seems odd to me and an outlier compared to all the other metrics.
    – Squirrel
    Commented Apr 5, 2022 at 17:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .