When you’re a company with more than 175 million active users and over 1.1 billion live listings, how do you maintain security, enable new innovation, and improve the user experience? eBay is the latest company to share its IT transformation.
At a time when many companies are just digging into microservices, FaaS, and a whole host of third-party cloud options to better focus their core business, massive technology companies like eBay are doing exactly the opposite.
eBay, the world’s leading online auctioneer, is transforming their IT to save money and create a custom, more efficient experience for both their employees and their customers. Interestingly, eBay is neither the first, nor likely the last, tech giant to bring more options in-house. But just because some massive tech companies are taking these big strides doesn’t mean every other company should follow suit. Let’s take a look.
Going public with their transformation
Daily, eBay handles 300 billion data queries and wrangles with a data footprint that’s above 500 petabytes – the equivalent of 2.5 million hours of videos. Because of this size and scale, worldwide and 24/7, eBay has to move massive amounts of data and manage the traffic in a way that allows a smooth user experience while still ensuring a secure, stable environment that’s flexible enough to encourage innovation and incorporate new technologies.
In fall of 2018, the company announced they were in the midst of a three-year plan they dubbed “replatforming”. The major activities in this replatforming include designing their own custom servers (forgoing third-party server options) and rolling out a new, decentralized strategy for their data centers. Like Facebook and Microsoft before them, eBay is relying on open sourcing to design their custom servers.
Choosing to decentralize their data center clusters is a serious, long-term move that requires significant investment. eBay’s primary servers have been located in Phoenix, the Salt Lake City area, and across Nevada. But the company is moving towards a Point of Presence (PoP) strategy, deploying physical infrastructure across the U.S. to be closer to end users – known as the edge.
This geographic expansion will reduce money spent on transporting data over long distances and networks, resulting in lowered latency and improved user experiences. This move to edge computing architecture could save up to 800 milliseconds of load time – which sounds like nothing but, in computing, is remarkable.
Of course, overhauling the entire physical infrastructure means a necessary redesign and rebuild of the entire logic and software stack that runs on the infrastructure – these must advance together. eBay intends to incorporate the most recent open source technologies in order to stay innovative, flexible, and number one in their business.
Despite relying on OpenStack for more than 90 percent of their cloud technology as recently as 2017, the company will mostly ditch OpenStack. Instead, they’ll opt for open-source technologies, like Kubernetes, an open-source container orchestration technology, and Apache Kafka, a stream processing platform that increases data handling and decreases latency, among others.
eBay says that by relying on open source technologies, they can further innovate – and they plan to share their advances back with the open source community. Also in the works is the creation of an in-house AI engine with functions like image search and computer vision in order to increase productivity.
The goal of this IT transformation? Ostensibly to improve the user experience and to promote productivity with their engineers and programmers. The replatformed system should be immensely more predictable, controlled, and flexible – the stuff engineers dream about. Plus, using open-source technologies means they can stay on the cutting edge, both tech- and business-wise. Doubtless, though, eBay’s bottom line will also reap significant gains.
Benefits of housing and customizing your cloud
A task like this, whether or not you call it replatforming, is certainly not for the faint-hearted. Indeed, most SMBs and even most large companies whose products aren’t immediately in the technology sphere are only at the beginning of embracing cloud technology.
But for those companies who have the need, scalability, time, manpower, and money to design and build their own servers, there are plenty of benefits. Optimizing hardware to your unique needs, decreasing money spend on third-party features you don’t use, relying less on third-party vendors (whose designs inherently limit your work environment), and taking advantage of competition with an open-source hardware design.
Indeed, this last benefit may be the most unique: when a huge company like eBay opts for open sourced hardware designs, it opens up the competitive market. Any supplies can make a realistic pitch to supply vast quantities of hardware – a huge win for the supplier – and the company gets the results of competition: lower cost and more reliability.
