#8 A new aggregation for a new horizon
Welcome to the 8th issue of the Pyth newsletter: HiFi for DeFi. This edition will please math enjoyers as well as the NFTs aficionados, so sit down and relish!
About:
The Pyth network is a vision coming to life: one that imagines the world of decentralized finance (DeFi) gaining comprehensive access to high-fidelity (HiFi) financial markets data securely and reliably.
In other words, the Pyth network is a specialized oracle network that focuses on sourcing continuous real-world and crypto-related market data originating off-chain and streaming it on-chain at sub-second speeds for smart contract consumption regardless of their blockchains.
This is an ambitious goal because financial market data is sufficiently unique: there are very few available sources, and those sources have very tightly controlled distributions. In addition, the oracle network needs to be able to combine such latency-sensitive data in way that optimizes not only accuracy but which increases security as well given how dependent many blockchain applications are on the accuracy of this type of data.
Website, Medium, Twitter & Discord
A New Aggregation Formula
One of the core tenants of Pyth is to rely on high-quality, first-party data as inputs for its aggregated output to ensure for the most robust and reliable price. Trash in, trash out; so with poor initial sources (and without beautiful aggregation logic), it is impossible to make the aggregated output of high quality. At Pyth, we have solved for both. First, we have secured, and continue to onboard, the world’s most prominent trading firms and exchanges to provide first-party data. On the second point, we have put forth a sophisticated aggregation formula we want to discuss here.
By now, you are probably aware that Pyth not only provides a live market price for assets but it also offers a confidence interval alongside it — read more on the Confidence Interval. This incredibly important and unique data point — mapping out the uncertainty one publisher has towards the provided price — actually enables the Pyth network to leverage a smarter aggregation method.
Now let’s dive into the new price aggregation proposal Pyth put forth on November 16th and looks to be deployed shortly on mainnet. You may find our full blog post here.
→ We want Pyth’s aggregation algorithm to have 3 properties:
Robust to manipulation
If most publishers are submitting a price of $100 and one publisher submits a price of $80, the aggregate price should remain near $100 and not be overly influenced by the single outlying price.
Aggregate price should appropriately weight data sources with different levels of accuracy
Pyth allows publishers to submit a confidence interval because they have varying levels of accuracy in observing the price of a product. This property can result in situations where one publisher reports a price of $101 +/- 1, and another reports $110 +/- 10. In these cases, we would like the aggregate price to be closer to $101 than $110.
Aggregate confidence interval should reflect the variation between publishers’ prices
In reality, there is no single price for any given product. Every product trades at a slightly different price around the world and so we must be able to reflect these variations.
How does the suggested aggregation algorithm achieve the 3 above properties?
The first step of the algorithm computes the aggregate price by giving each publisher three votes — one vote at their price and one vote at each of their price +/- their confidence interval — then taking the median of all the votes. The second step computes the distance from the aggregate price to the 25th and 75th percentiles of the votes, then selects the larger of the two as the aggregate confidence interval. Now, let’s put everything together and visualize it with 4 scenarios.
In the following graphs, the red star depicts the aggregate price and the bold red line depicts the aggregate confidence interval. The grey circles represent the 25th and 75th percentiles of the votes — the further one of these from the aggregate price determines the confidence interval’s width.
Scenario 1
One “confident” publisher (tight CI) is an outlier to the cohort but does not impact the final aggregated price. Its only impact will be a greater aggregated confidence interval that could highlight a price dislocation of the asset on different venues.
Scenario 2
It demonstrates how publishers with tighter confidence intervals (while having overlapping quotes with the rest of the cohort) can exert greater influence over the location of the aggregate price.
Scenario 3
It features publishers in overall “agreement” regarding the price and their relative uncertainty towards it. The final result shows that the aggregate confidence interval accounts for each publishers’ CI and gives identical results to the ordinary median.
Scenario 4
Publishers publish distinct prices with non-overlapping CI. In this case, all votes of a single publisher will be adjacent in the sorted list and will be treated as a single vote.
Even if we would not be in any of the above scenarios mentioned, the aggregate price will always lie within the 25th-75th percentile of the publisher’s prices.
In addition, we’re working on a staking system for publishers that incentivizes them to provide accurate data, and in that system, each publisher will have a varying amount of stake. All of the results also hold for stake weights if we simply replace the % of publishers with the % of stake controlled.
You will discover more on this topic as the whitepaper gets released (SoonTM) to the public.
Miami Hacker House Results
Presented in our last Substack, Pyth did participate in the Solana Miami Hacker House, which took place from December 1st to the 7th, with the emission of a bounty.
Pyth NFT Bounty for 50K USDC 👇
Description: Build a Pyth oracle that publishes floor price and average price for any mint on the top six marketplaces on Solana (Solanart, ME, Alpha Art, Exchange.art, Digital Eyes, and FTX.us).
Specifics: One Pyth Oracle struct per mint, presents latest floor price and latest average price, with an enum class for the marketplace. The ETL should produce updates to these oracles at least once per day.
And as the Hacker House was closing its doors, we announced the NFT Bounty winners: Scott Martin & @if_name_main with their Quokka proposal. They submitted a complete, well-documented solution that read NFT prices from all 6 major exchanges and pushed them to Pyth.
What's next with NFTs and for this bounty?
Since their creation and recent mainstream adoption, the term "NFT" or Non-Fungible Token has been predominantly used when referring to digital art pieces, with only sporadic mention of their even more complex functionalities (like fractionalization) that enables more advanced financial products.
We will dive into a future blog on the latter: "Financial NFTs", and instead, focus first on the initial premise that NFTs are "similar" to art pieces (but digital) and so carry a tangible value. Regardless of an artist's name, fame, or even the beauty of the artwork itself, such item is worth something as long as someone wants to acquire it. With this in mind, and the crypto (overly financial) mentality was born the idea that NFTs could/should serve as collateral for any financial operations existing.
What does it mean in practice? Imagine you are Ezra (or David) Nahmad — whose collection estimate ranges around $3B — and are in dire need of cash. Would you sell one of your precious paintings for the money outright (and missing the potential upside of its valuation or solely miss gazing at this masterpiece you own) or — if you could — would you offer to someone, you trust, your painting as collateral for cash? Valued at an agreed amount (could be the latest highest bid for instance), with a (variable or fixed) interest to be paid as long as the operation lasts, you now have found an agreement between 2 parties. Ezra gets its deeply wanted cash without outright renouncing the ownership of the supplied painting, while his creditor earns interest on the risk the latter takes, everybody wins.
Now replace the term "painting" with "NFT" and you have made those digital art pieces some kind of "productive" assets and even more valuable.
The Pyth NFT bounty was the 1st step towards enabling DeFi applications to incorporate NFTs, similarly to other more traditional crypto assets, in their offering.
Now to make this a reality, as we have been and will continue doing for crypto, equities, FX, and commodities market data, we must onboard under one roof all possible first-party data providers to contribute their proprietary inputs into to the network that will, in turn, produce a more robust output and of higher quality. So watch out for future announcements and new partners joining the Pyth quest to offer the highest quality data possible to anyone for free.
In the meantime and if you are interested to know more on this topic, we highly suggest you to take a look at the amazing NFT articles from Paradigm: Floor Perps or RICKS.
Get ready for future hacker houses, hackathon events, and Pyth workshops in 2022!
New Publishers
This month we welcome two new data providers onto the Pyth network, as the roster continues to scale with the best minds in financial markets. Quality over quantity and please reach out if you think you have strong first party data to contribute!
Talos is a leading institutional-grade infrastructure technology provider that supports the full lifecycle of digital asset trading, from price discovery to execution through to settlement.
Engineered by a team with unmatched experience in building institutional trading systems, the platform is trusted by many of the world’s largest market participants, using it to forge bi-lateral relationships with all members of the crypto trading ecosystem.
Founded in 2018, Talos has rapidly scaled its business since announcing its platform last October. In May, Talos announced the completion of a $40 million Series A investment round led by Andreessen Horowitz and investments from PayPal Ventures, Fidelity Investments, Galaxy Digital, Elefund, Illuminate Financial, and STEADFAST Capital Ventures.
JST Capital is a global digital asset financial services firm servicing institutional investors, blockchain companies, banks, and broker-dealers.
With operations in the United States and Singapore, the Company provides sophisticated trade execution, asset optimization, treasury services, and consulting services to facilitate liquidity.
JST entered the crypto markets as early as 2014 when they began to make markets for one of the early blockchain projects. Leveraging that experience, JST was launched in 2018 to provide a full suite of traditional financial services to institutions in the digital asset market.
Once again, the Pyth network has been and will continue to onboard some of the biggest trading firms & exchanges as data providers, as they own data with granularity and quality that is traditionally guarded by the largest, centralized gatekeepers of the financial world.
As so many prominent partners join us in our quest to bring HiFi to DeFi, it only felt normal to showcase this incredible roster on our website with our Publishers tab.
That is all for our 8th newsletter — Thank you for reading, and don’t forget to subscribe and share!
We can’t wait to hear what you think! Feel free to join the Pyth Discord server, follow Pyth on Twitter (and our new Chinese Twitter), join the Telegram and the Substack or the Medium to learn more, and ask any questions you may have.