Grant Efficacy Analysis and Dashboard

Abstract:

An analysis of the efficacy of grants awarded to projects by the Apecoin Foundation. Efficacy is based on the project’s contribution to the Apecoin ecosystem in terms of onchain activity (users, fees, etc.). The analysis will be presented in a report and a dashboard.

Benefit to the ApeCoin Ecosystem:

This requirement stemmed from this tweet by 0xQuit, who highlighted that there are a number of grantees who have nothing to show for the amount they were granted. However, there is no adequate tool or analysis to assess – grant amount awarded against value accrued back to the Apecoin community.

This proposal aims to address this quantitatively through a report analysing the projects that received a grant, and how much value was accrued back to Apecoin.

Key areas that will be addressed are:

  1. Analysing growth in the user base and cross-usage of protocols amongst users.
  2. Analysing changes in user behaviour and frequency: Avg no. of transactions per user, fee per user
  3. Analysing the cost of acquiring users through CAC and the payback period.
  4. Calculate the lifetime value (LTV) of users and LTV / CAC ratio.
  5. Analysing the quality of users through retention cohorts and repeat ratios.
  6. Build and analyse the behaviour of various Grantee segments (By scale, size and business category). Segments will be made by dividing Grantees into groups with similar characteristics or behaviours.
  7. Benchmarking of Apecoin ecosystem metrics with comparable protocols across metrics and protocol activity. This will enable us to outline areas where Apecoin is outperforming and where it is lacking.

Based on this analysis we aim to answer key questions of capital efficiency. Incentives, in the form of grants, constitute a significant portion of the foundation’s spend and allocating them for the right projects is most important to the long-term growth of the ecosystem.

Our work will answer the following questions:

  1. What is the ideal set of projects for the DAO to fund and support through the grants based on their Sector, Scale and Traction?
  2. Understand if the DAO should double down on established grantees or if DAO should make bets on newer projects
  3. What is the optimal size of the grant to generate maximum impact?
  4. What is the ROI of various segments, and which were the best-performing and worst-performing grantees?
  5. For a sample set of sectors, analyse why they underperformed or overperformed through analysis and discussions with Grantees/community members.
  6. Create a leaderboard of performers based on varied sets of metrics e.g. which segments created the most sticky users (best retention cohorts from M1 to M30), which segments are driving daily active user growth, and which segments drive maximum TVL?
  7. What sectors should the DAO bring in and support through grant funding based on the sector performance on Apecoin versus the sector performance on another ecosystem?
  8. What were the unique incentivization strategies implemented by the grantees to attract new and recurring users?

Methodology

We will segment grantees based on different parameters and will conduct pre- and post-analysis of those segments. We aim to start with the recently concluded GWG x UMA On-Chain Small Grants Program, and earlier grant programs to present a comprehensive view of Apecoin’s grant program analysis. Here, we aim to divide the grantees based on the same categories of the grant program:

  1. Education
  2. Social Good
  3. IRL Events

Based on the size of the grant:

  1. Low Spend: Below 2,500 APE
  2. Medium Spend: 2,500 to 5,000 APE
  3. High Spend: 5,000 to 8,500 APE

(This will be scaled for larger grant programs accordingly)

Based on the project’s size/scale/recency (when was the project launched). According to the predefined quantitative methodology, projects will be divided into ‘Large’, ‘Medium ’, and ‘Small’ categories.
The methodology for the categorization will be documented and presented separately.

Performance Analysis

To calculate performance of the grant program, both as a whole and for individual grant awardees, we will calculate:

  1. Cumulative users: Calculate the total users who have used Apechain
  2. Active users: Monthly breakdown of new vs existing users, showing how many new users were acquired in each week
    • Daily and monthly active users:
    • Daily Active Users (DAU)
    • Monthly Active Users (MAU)
    • Ratio of DAU / MAU
    • Breakdown of DAU, MAU & DAU/MAU by new vs existing users
    • Retention cohorts for Apecoin before and after the grant for each grantee and overall for the Apecoin ecosystem show retention at 1W, 2W, 4W, and 8W.
  3. Frequency: Analyse the avg number of transactions per week per user for Apecoin, and on Apechain during the grant period, and after it
  4. Economics: Analyse trends in the avg fee per transaction
  5. Wallet activity: User activity by existing vs new wallets, and age of wallet (did they hold APE before vs after the program)
  6. Acquisition cost vs value to Apecoin: CAC (grant program cost vs number of new wallets onboarded) to the LTV (gas contribution of a wallet)
    • CAC to LTV ratio
    • break-even ratio (at what level of activity and retention will Apecoin recover grant money in fees)

Presentation

  • Dashboard: A public dashboard for the metrics we will analyze and a related document of methodologies and definitions.
  • Report: A report containing insights and conclusions of our research and analysis.

Team Background

PYOR is a crypto data analytics company backed by Castle Island and Coinbase Ventures. We specialize in measuring the business performance of protocols/chains. We work with large institutional investors and protocols in the web3 space. We’ve worked with Ribbit Capital, M31 Capital, Tezos, Compound, ICP, Swell, QuickSwap, etc.

Past work

  • We work extensively with large institutional customers supporting their Crypto analytics and research needs. Our customers include Ribbit Capital and M31 Capital amongst others.
  • We extensively work with protocols to build custom solutions tailored to their needs. Some of the Protocols/chains we have worked with include – Livepeer, Compound, Tezos, ICP, Swell, QuickSwap, and Osmosis among others.
  • We have built an interface X-ray for institutional investors to give them an in-depth look into the business and financial metrics. Our platform includes metrics around retention, LTV, user activity, stablecoin volumes, etc.

The following is analysis we’ve conducted on similar protocols:



Requested Budget & Cost breakdown

We request a budget of APE 80k for this project, divided as follows:

Cost
On-chain data data collection 8,000 APE
Infra cost for data modelling and analysis 19,200 APE
Team cost (3 Research analysts and 1 data scientist for 3 months) 43,200 APE
Engineering efforts for dashboard - Front end and back end 9,600 APE

I’m not really seeing how this proposal quantitatively determines value accrued back to the Apecoin community based on previous approved AIPs.

It just seems you are determining various different data points on Apecoin and Apechain.

This line:

Appears to be the only one focusing specifically on authors of AIPs, and even then it becomes more nuanced as each AIP is different from the others with different objectives/goals.

These are great questions, but I’m not seeing how you plan on answering these with what you’ve mentioned in your “Key areas that will be addressed” nor with what your calculating in “Performance Analysis”.

Why are you looking to start with On-Chain Small Grants Program, when this isn’t what 0xQuit was referring too.

I’m really not seeing the link between determining these data points on Apechain and how they correlate to an approved AIP.

Overall, this just seems like a dashboard of various different activity levels for Apechain.

1 Like

Hi @AakashPYOR,

The community feedback period for your proposal would be ending in less than 24 hours.

  • If you’re content with the feedback received, your next steps are to finalize your proposal using the AIP Draft Template.

  • A moderator will reach out to the author to finalize the AIP Draft. Upon receipt of the final Draft, we will review and provide instructions on the next steps.

  • Are you ready to proceed to the next phase or do you wish to extend community discussion for another 7 days?

We look forward to hearing from you.

-@Facilitators

1 Like

Thanks for your questions @LiveFast9986. I am replying to them in order:

To clarify, the goal of this grant efficacy analysis is to see value accrued by onchain grantees to Apecoin based on core metrics like – fee increases during grantee’s project operations, users increase during grantee’s project operations, increase in trading volume (or other core grant specific metric) during grantee’s project operations. Beyond this, we aim to go deeper into each metric (fee/user/volume) to see – growth, new activity, cost of acq vs value, etc.


We won’t be determining different data points. Rather, we’ll be looking at the change in these data points during and after a grant period to assess the positive impact as a whole.


I agree. It’s hard to categorize AIPs based on segments because of the varying nature of each AIP grant. However, grant amount (in APE) is an easier classification. Hence, we will first classify grants based on grant size and will seek community feedback for category-based classification.


We plan on answering these questions based on the data. For instance:

  1. What is the ideal set of projects for the DAO to fund and support through the grants based on their Sector, Scale and Traction?

Based on the grants (divided between sector, scale, and traction) that generated the highest positive change in fees, users, and volume.

  1. What is the optimal size of the grant to generate maximum impact?

Based on the dollar value of the grants released to the dollar value of the fee generated.


We plan to engage with the community for a full-scale grants analysis over the last two years. However, I was only able to find the On-Chain Small Grants Program on the forum. If you have the link to a larger grant program, I will include the program and its classification here. For the record, we aim to conduct a grant efficacy of all material grants in the last two years (not limited to the On-Chain Small Grants Program).


As mentioned earlier, the output of this will be a – dashboard and report – specifically for Apechain’s grant programs. The metrics presented and analysed will be for grants and only insofar as they effected (positively or negatively) the core metrics of Apechain. The dashboard will not have other general metrcis of the chain.

This topic was automatically closed after 7 days. New replies are no longer allowed.