HCAHD information source - Apache Hadoop Developer Updated: 2024

Voltaire Hitachi Ltd.

Voltaire executives declined to disclose the size of Hitachi's investment, but said Hitachi intended to work with Voltaire on development of new InfiniBand products.

Hitachi's interest in the company comes from Voltaire's development of a complete InfiniBand router, including hardware, software and drivers, said Arun Jain, vice president of marketing at Voltaire, based here and in Israel.

Jain said other developers have focused on only one part of the solution. He also said Voltaire is already shipping a 1x InfiniBand router, which helps connect InfiniBand server and storage components to standard TCP/IP networks.

Jain said sales are very slow because the market is waiting for 4x equipment. Voltaire plans to have a 4x router in evaluation next quarter and begin shipments early next year, he said.

Jain admitted that backpedaling by Intel and Microsoft, lack of silicon for InfiniBand and the soft economy have held back the adoption of InfiniBand. "But our partners are saying they plan to put InfiniBand in their products quickly," he said.

Voltaire currently has a limited sales force and is awaiting the availability of its 4x router before developing an indirect sales channel, Jain said. He expects most of the company's sales to go through the channel when that happens.

Hitachi Vantara, a leading developer of IoT and data analytics technologies, wants to integrate new automated data management via a planned acquisition of Waterline Data, a startup developer of automated data cataloging technology.

Data management and analytics technology developer Hitachi Vantara Wednesday said it plans to acquire Waterline Data, a startup developer of intelligent data cataloging technology.

With the acquisition, Waterline Data's unique data cataloging technology will be integrated with Hitachi Vantara's data operations technology to automate management of data across all of a business' data sets, said Lothar Schubert, head of product marketing for Santa Clara, Calif.-based Hitachi Vantara.

[Related: Hitachi Vantara's 5 Biggest Announcements At Next 2019]

Waterline Data will bring to Hitachi Vantara strong and differentiated capabilities in data cataloging, particularly to Hitachi Vantara's Lumada big data and IoT platform and its Pentaho data management and analytics platform, Schubert told CRN.

"It provides common metadata for our Lumada portfolio and extends and differentiates our data operations," he said.

Waterline Data has developed its own intellectual property it calls Data Fingerprinting that helps with the automation, discovery and classification of data, Schubert said.

"Waterline Data uses a combination of AI and rules-based systems to identify data," he said. "It crawls data and learns its metadata. So it can see if data is an insurance claim, a Social Security number, or a customer's data. And it can catalog data across terabytes of capacity. It helps in discovering, tagging and managing data, which is a big deal in GDPR and the California Consumer Privacy Act."

Waterline Data is a startup, but already has products and an existing customer base, Schubert said. "So it's a proven technology already," he said.

The company's technology integrates with most standard and open-source databases, hyperscaler and data warehousing technologies, Schubert said.

"Customers tend to be heterogeneous in their data technology," he said. "They need to connect across a heterogeneous infrastructure."

Hitachi Vantara plans to continue making Waterline Data's technology available as a stand-alone offering while working to integrate that technology into its Lumada and Pentaho platforms, Schubert said. "We will leave no customer behind," he said.

Schubert declined to talk about Waterline Data's size or the financial terms of the acquisition.

• Demand for open source developers with container expertise is soaring, with 57% of hiring managers prioritizing this expertise in 2018, up from 27% last year.
• Hiring managers are choosing to invest in and scale up new hires and existing employees by investing in training versus spending on external consultants.
• Be active in the open source community and contribute the highest quality code and content possible.

The 2018 Open Source Jobs Report published by The Linux Foundation and Dice.com provides many useful insights into which skills are the most marketable, the technologies most affecting hiring decisions, and which incentives are most effective for retaining open source talent. Taken together the many insights in the study provide a useful roadmap for recently-graduated students and experienced open source, developers, and technical professionals. The study is based on a survey of over 750 hiring managers representing a cross-section of corporations, small & medium businesses (SMBs), government agencies and staffing firms worldwide in addition to 6,500 open source professionals. Additional details regarding the methodology can be found in the report downloadable here (PDF, 14 pp., opt-in).  The following findings show how strong demand is for developers with open source expertise and which skills are in the most demand.

• The number of hiring managers seeking Linux talent soared in 2018 to 80% up from 65% in 2017, making this the most in-demand open source skill. Cloud technology experts are the second most in demand at 64%, followed by security and web technologies (49% each). Networking (46%) and containers (44%) are the six areas managers are prioritizing in their recruiting and hiring. Six in ten (62%) of open source professionals rank containers as the fastest growing area of 2018. 48% of open source developers believe Artificial intelligence and machine learning is growing the fastest followed by security (43%).

• Knowledge of cloud, container and security technologies most affect who gets an offer according to hiring managers. Having deep expertise in Linux in addition to experience working with cloud, containers, security, and networking is an effective career strategy for getting an open source developer job and progressing in a technical career. 66% of hiring managers cited cloud computing as the technology that most affects their hiring decisions, followed by containers (57%), security (49%) and networking (47%).

• Hiring managers are choosing to invest in and scale up new hires and existing employees by investing in training versus spending on external consultants.  55% of employers are now also offering to pay for employee certifications, up from 47% in 2017 and only 34% in 2016. Global Knowledge recently completed a study of which certifications are the most valuable, and they are summarized in the post, 15 Top Paying Certifications For 2018.  55% of employers are willing to pay for certifications today, up from 47% last year and 34% in 2016. 42% are using training as an incentive to retain employees, up from 33% last year and 26% in 2016.

• Despite employers prioritizing pay to recruit open source developers, 65% are in the field to work with new technologies, 64% for the freedom of open source and 62% because they have a passion for the field. Open source developers who are the most likely to find a new job translate their passion for the field into shareable code and content. Getting a dream job as an open source developer starts by becoming part of the community and actively contributing as much as you can. Creating and sharing open source code via GitHub and other means, blogging and sharing what you’ve learned is a great way to stand out from other applicants. I recently had lunch with a good friend who is looking for an open source developer job and this person’s GitHub is getting a ton of downloads from a recent open source app written for data visualization. And a presentation given at a recent conference is leading to interview opportunities. The open source community is very reciprocal, and it’s a great idea to share the highest quality code and content there, and can lead to job interviews.
• Open source skills are lucrative, and in high demand, with 87% of open source developers crediting their expertise and continual learning of new apps, languages, and tools as the reason they are advancing in their careers. 52% of hiring managers say they will hire more open source professionals in the next six months than they did in the previous six months, crediting their company’s growth as a result of a strong national & global economy. 60% of hiring managers say the number of available positions for open source developers in increasing at a faster pace than overall open positions. The market value of DevOps skills grew an average 7.1% during the past six months according to analyst firm Foote Partners.
• There’s an 18% gap between what employees say is progress on diversity versus employers. Only 52% of open source developers say the level of effort at attaining greater diversity is effective versus 70% of employers. Employers need to consider how they can use advanced hiring systems beyond Applicant Tracking Systems (ATS) to get beyond the potential for conscious and unconscious bias in hiring decisions. A previous post, How to Close The Talent Gap With Machine Learning, provides insights into how employers can remove biases from hiring decisions and evaluate candidates on talent alone.

Against a backdrop of increasing concerns about software security, the U.S. government has recently taken a series of actions to Boost software security. This effort started with the White House Executive Order 14028 on Improving the Nation’s Cybersecurity and was followed by two Office of Management and Budget memorandums, M-22-18 and M-23-16, that set schedules and requirements for security compliance. The White House released a National Cybersecurity Strategy and then followed up with an implementation plan, with many of the elements of the plan already underway.

Each of these actions include details about how the government and its suppliers can Boost the security of the open source software components that make up a large percentage of the code being used in government applications. But even beyond these government-wide actions, two recent developments show the government is investing in modern, proactive strategies for improving open source software security.

1. The first U.S. government open source program office

The Center for Medicaid and Medicare Services (CMS) recently established the first U.S. government open source program office (OSPO), where the agency is implementing a developer-minded, private-sector-styled strategy to modernize their approach to open source software. The designation of its first dedicated open source program office is an encouraging signal that the federal government recognizes the strategic value of open source and the innovation it can bring to our government agencies.

As Andrea Fletcher, chief digital strategy officer for CMS, explained it: “We already have a lot of really fantastic open source programs… [and are focused on] pushing these programs forward and how we release our software and our code to the greater healthcare ecosystem… We’re pushing this out over the next couple of years to see what it looks like for an agency to have policies around inbounding and outbounding code.”

Questions for readers:

• Has your organization considered following this lead and building an open source program office to manage your open source software policies, including both how you manage contributing to open source externally and how you bring in and manage the open source components your organization is already using? As new government cybersecurity requirements take effect, an open source program office might be a good centralized place from which to organize your efforts to ensure the open source built into your applications stays in compliance with the requirements under M-22-18 and M-23-16.
• Who in your organization is tasked with staying up to date on government cybersecurity regulations impacting open source software, and how can you stay in step with the rapidly evolving requirements and deadlines?
• Which part of the organization will be taking the lead on the secure software development attestation requirements under M-22-18 and M-22-16?

2. Investing in the security of the open source software supply chain

The Office of the National Cyber Director recently released an RFI on Open-Source Software Security and Memory-Safe Programming Languages, seeking ideas from the public and private sector on how to use government resources to invest in improving open source software security. 

One particularly interesting section of the RFI requested ideas around incentives for securing the open source ecosystem.

The core challenge of securing the open source software ecosystem is that it is unlike any other supply chain that is so critical to the global economy because the “suppliers” are largely independent and often unpaid developers (usually called “maintainers”). 

A recent study by Tidelift found that 60% of open source software maintainers described themselves as unpaid hobbyists. The reason the government is focused on maintainer incentives is because unpaid hobbyist maintainers often lack the time and motivation to implement the secure development practices that government and industry require. 

So it is significant to see this RFI looking to address incentives for improving open source software security—potentially by paying maintainers to do the work to implement secure development practices like those recommended in the NIST Secure Software Development Framework.

Questions for readers:

• Are you aware of the NIST Secure Software Development Framework, and the best practices it outlines for the development of software in and for the federal government?
• Do you know whether the open source software being used in the applications you sell to the government follows the practices outlined in the NIST Secure Software Development Framework or whether the project maintainers have the time and incentives to do this work? 
• Without this information, how will you ensure your software complies with new government cybersecurity requirements potentially impacting the open source components you use?

It’s becoming clear that many within the U.S. government, in agencies like CISA, ONCD, CMS, NIST, and OMB, are applying modern thinking to how they manage the security risks of building with open source so they can still take advantage of the enormous innovative potential it provides. 

Is your organization following the lead? Are you thinking about centralizing how you manage your open source policies and practices like CMS is? Are you keeping track of emerging cybersecurity policies and standards impacting open source so you can ensure you are following security best practices NIST recommends, and not endangering your government revenue by missing key deadlines and requirements from OMB? And are you thinking proactively about the volunteer suppliers who you count on for the open source code you use, like ONCD is, and how you can ensure the maintainers who create it are incented to keep it secure into the future?

Open source can be a powerful and positive innovative force when managed effectively, and a liability when not. These are the types of questions you should be asking in order to stay in tune with leading government initiatives that will help point the way to your organization’s success with open source.

Rob Wickham is vice president of public sector at Tidelift and a 20 year veteran of supporting the U.S. federal government by delivering innovative technology solutions that address critical capabilities in the areas of cybersecurity, DevSecOps, hybrid architectures, and zero trust. He is a frequent panel participant speaking on syllabus including zero trust, identity and access management, and emerging technology trends like software supply chain security, and vulnerability prevention.

The Metabo and Hitachi brands were hardly strangers before the 2018 rebranding. In fact, the Hitachi Group added the German Metabo company to its portfolio a couple of years before it offloaded its power tools division in 2017. Though Hitachi Power Tools was quickly re-branded as Metabo HPT after that sale, it operated as a separate entity from Metabo until Koki America merged the two in North American markets in 2022. 

All the corporate maneuvering didn't amount to much more than a name change from the Hitachi Power Tools team. Even the history page on Metabo HPT's website confirms that the rebrand was intended to change nothing about the Hitachi brand but its name, with the company remaining dedicated to producing the same high-quality tools and well-loved tool kits it always has. 

So little has changed since the re-brand that you can still use Hitachi's interchangeable MultiVolt batteries in new Metabo HPT devices. That should come as a comfort to longtime Hitachi Power Tools users.

There has, however, been some confusion between Metabo and Metabo HPT in North America and abroad amid the rebrands and mergers. Despite the NA name merger, it seems there's still not much in the way of compatibility between Metabo and Metabo HPT tools since they continue to operate on different battery systems. It remains to be seen if that will ever change or merge together.

OpenAI plans to launch a store for GPTs, custom apps based on its text-generating AI models (e.g. GPT-4), sometime in the coming week.

In an email viewed by TechCrunch, OpenAI said that developers building GPTs will have to review the company's updated usage policies and GPT brand guidelines to ensure that their GPTs are compliant before they're eligible for listing in the store -- aptly called the GPT Store. They'll also have to verify their user profile and ensure that their GPTs are published as "public."

The GPT Store was announced last year during OpenAI's first annual developer conference, DevDay, but delayed in December -- almost certainly due to the leadership shakeup that occurred in November, just after the initial announcement. (The short version of the story is, CEO Sam Altman was forced out by OpenAI's board of directors and then -- after investors and employees panicked -- brought back on with a new board in place.)

GPTs don’t require coding experience and can be as simple or complex as a developer wishes. For example, a GPT can be trained on a cookbook collection so that it can answer questions about ingredients for a specific recipe. Or a GPT could ingest a company’s proprietary codebases so that developers can check their style or generate code in line with best practices.

https://platform.twitter.com/widgets.js

Developers can simply type the capabilities they want their GPT to offer in plain language and OpenAI's GPT-building tool, GPT Builder, will attempt to make an AI-powered chatbot to perform those. Since shortly after DevDay, developers have been able to make and share GPTs with others via the ChatGPT website directly but not publicly list them.

Still unclear is whether the GPT Store will launch with a revenue-sharing scheme of any sort. As of November, Altman and CTO Mira Murati told my colleague Devin Coldewey that there wasn't a firm plan for GPT monetization, and the email about the GPT Store's coming launch makes no mention of what developers can expect on the payments front -- if anything.

An OpenAI spokesperson told TechCrunch that more will be revealed next week.

As I wrote for TechCrunch's semiregular AI newsletter a while back, OpenAI’s shift from AI model provider to platform has been an interesting one to be sure — but not exactly unanticipated. The startup telegraphed its ambitions in March with the launch of plug-ins for ChatGPT, its AI-powered chatbot, which brought third parties into OpenAI’s model ecosystem for the first time.

GPTs effectively democratize generative AI app creation -- at least for apps that use OpenAI’s family of models. In fact, GPTs could kill consultancies whose business models revolve around building what are essentially GPTs for customers.

Is that a good thing? I’d argue not necessarily. But we'll have to wait to see how it all plays out.