HDFS Erasure Coding

Erasure coding provides redundancy by breaking the data into smaller pieces and storing the pieces in different locations. The key is that you can recover the data from any combination of a smaller number of those pieces.

Erasure coding technology

Erasure coding consumes less storage than replication and allows you to configure the robustness of your system (how many device failures you can recover from) with a very minimum storage cost.Erasure coding is useful when dealing with large quantities of data and any applications or systems that need to tolerate failures, e.g. disk array systems, object stores and archival storage.[su_spacer]The erasure coding framework in Hadoop Distributed File System is a key new feature and is also one of the driving features for releasing Hadoop 3.0.0. It can reduce storage costs typically by 50% compared to 3x replication while maintaining the same or better durability. Additionally, erasure coding adds minimal overhead, and can improve read and write performance in some conditions.[su_spacer]Erasure codes reduce storage by half, but speed of processing is crucial for a fast service to users. Chocolate Cloud provides its customers with HDFS-EC Plugin which is faster than Intel’s ISA-L in most configurations and is the only library to provide hardware acceleration on both Intel and ARM processors. Our goal is to exploit the power of erasure coding for delivering the ultimate performance and reliability to our customers.  Want to know more about Chocolate Cloud’s HDFS Plugin? You are always welcome to contact us at info@chocolate-cloud.cc and we will answer all your questions.

We demonstrated our first version of the Secure Object Based Storage using Intel SGX in Brazil

During H2020’s SecureCloud project review meeting, our team presented a demonstration of a first version of our novel and secure Object Based Storage solution for the Cloud using Intel SGX capabilities to create enclaves within the Intel chip itself to guarantee that data is not only communicated securely to the Cloud, but also that its processing and storage maintains the highest level of confidentiality and privacy. This is critical for the highly sensitive data streams from emerging Smart City and Big Data Cloud applications and that the SecureCloud project aims to tackle.

We are currently hosting a version of the demonstration with a public API as part of one of our partners’ infrastructure. If you are interested in trying out this first version of the system, please contact us.

The SecureCloud Consortium held its First Review Meeting on February 13-17, 2017 at the “Hotel Armação” in Porto de Galinhas, Brazil, where each industrial and academic partner provided the highlights of the achieved results in the first year of the project. In particular, there was a key focus on the development status of the components that will be used for the implementation of the final SecureCloud platform.

Cloud Backup product with partner O&O Software is shipping as Beta

Chocolate Cloud has provided its partner, O&O Software, the key component for Secure Multi-Cloud data management.
Together we have built a Secure Cloud Backup solution for Small and Medium Businesses to act as a Tape Replacement solution.

We call this product the (CloudCuber/Box) and it is currently shipping in Beta to selected customers with a multi-site cloud service provided by Cloud & Heatin Germany. We are excited to see our technology in use and in the market!

Chocolate Cloud CEO is interviewed regarding the participation of the company in the SecureCloud project

The interview appears in the Danish magazine “EU-Information” No. 3, pp. 5, December 2016

To go or not to go Horizon 2020?

By Sasha Bermann, EuroCenter, Board of Research and Innovation

When does it make sense for a private company to enter Horizon 2020?

Chocolate Cloud CEO Daniel Lucani, gives an answer to the question: When does the Horizon 2020 program make sense and when is it a ‘no go’­?

The company Chocolate Cloud ApS has secured its first Horizon 2020 project, SecureCloud, but this is not Daniel Lucani’s first Horizon 2020 project. The CEO’s CV contains other research and innovation projects, as is the case with some of his colleagues in Chocolate Cloud. Therefore, he knows the problem of EU funding very well from a company’s perspective.

Why did you decide to apply for Horizon 2020 funds?

I did so because I saw an opportunity to develop the company’s core competences, which would be useful in terms of our medium-term product development. In addition, I also saw the possibility of expanding our international network with industry and university actors.

What did you take into consideration before you made the decision to apply for Horizon 2020 funds?

It was imperative that Horizon 2020 fits into the company’s commercial roadmap, so that the Horizon 2020 activities would strengthen our business. The resources we use on the Horizon 2020 project are resources which we cannot apply to other commercial activities. I knew in advance, because of my previous experience with EU-funded projects, that the project would definitely require some resources. For a partner – not a coordinator – it takes about 100 hours to review the application process. The life of the project is also very resource-intensive. Here I refer not only to the technical but also to the coordination within the consortium and to the reporting to the European Commission.

 What would have changed your decision or what could have been the so-called ‘no go’?

You cannot go on with an application if the Horizon 2020 announcement does not fit into the overall strategy of the company. If that was the case for Chocolate Cloud, we wouldn’t have been in SecureCloud today. Another important factor is the composition of the consortium. In my view, a consortium should be composed of a mix of academic participants, industry participants and end users, and that is what we got in SecureCloud. We do not go into projects where the consortium’s inner circle does not know each other.

What is the added value of a Horizon 2020 project versus a Danish funded project?

The big difference is the international dimension of a Horizon 2020 project, where you typically have partners from a number of different countries in the project. It is ideal if the market you are looking for is international instead of your home market. At the same time, I would like to point out that the international dimension, of course, also challenges the cooperation within the consortium simply because of the geographical distance.

What strategic considerations did the Horizon 2020 project lead you to?

The SecureCloud project has provided us with a thorough discussion of our core technology with our end users and partners. It has given us a better understanding of their needs. At the same time, these discussions have had a significant influence on our vision and future strategy.

HINTS

Daniel Lucani’s recommendations for private companies considering Horizon 2020:

  • Have a common vision for the project in the consortium.
  • Know the key partners in the consortium: The stronger the relationship between the partners, the better the cooperation will work.
  • There must be some experienced partners in the consortium, especially the coordinator should have experience. Integrate some academic partners in the consortium, because they are used to writing applications to Horizon 2020 and, thus, have a very valuable contribution to the process. They also have a supporting organization that can help run the project when it’s in progress.
  • Good balance in the consortium: A good consortium should include both academic partners, industry partners and end users.
  • Be sure to give the project a complete schedule to suit your company’s commercial roadmap: As a company, you should learn things that you can commercialize later. This happens when you keep in touch with the development of the project.
  • Be serious about your commitment to the project: Be aware that the project will swallow some resources, so do not go into the project without a clear idea of ​​what you want to get out of it – for example a project result that you can commercialize later.
  • Be realistic in your expectations of what the project can do for you: Horizon 2020 brings together different teams, it does not create a product, but it paves the way for you to create a product based on project results.
  • Be active in the application process: this gives you the ability to influence the design of the application, ensuring that it is relevant and valuable to you.

Our CEO presented at the Danish Industry (Dansk Industri) meeting

The Danish Industry meeting, which took place on June 8, 2016, was a big opportunity for entrepreneurs to come and hear how their company can do innovation projects with other European companies as part of the H2020 framework and what are the opportunities available for them. During the day, the participants in the meeting could hear what different Danish companies have gained from participating in European projects and what is their formula for success.

Our CEO, Daniel Lucani, was invited to share his experience and knowledge Chocolate Cloud has gained while working on the H2020 Secure Cloud Project. The project focuses on solving the issues that arise in IT security in the cloud, particularly, if you cannot or will not trust the infrastructure software (Virtual machines, Operating System running in the machines, etc). This is a problem area that is becoming more and more critical as society is being digitized. “Big data in the cloud” was the business case which Daniel Lucani presented at the meeting. He gave more insights about the Secure Cloud Project, Chocolate Cloud’s technology and the lessons learned over the years.

Click here to find out more about the presentation.