If you are anything like me, you have absolutely no idea what a time of flight camera is, but you heard about it reading about the specs of the Huawei P30 Pro. What is a time of flight camera?  Why did this phone have it?  Did I need it?  I mean obviously I need it, but do I need need it?

Naturally, wanting to know what this new piece of tech is, I wondered off to the big brother search engine and fired away time of flight sensor

I was immediately greeted with a bunch of time of flight sensors for sale, and even more companies offering them for machine learning and vision.  Then a quick scroll further, I was corrected and told I really wanted to be looking for time-of-flight cameras, not sensors.

What is Time of Flight?

A quick query adjustment had me realizing how little I actually knew about time-of-flight cameras, or even the concept of time of flight.  So I decided it was time to dig out my science cap, and figure out what this term time of flight meant.

Unfortunately, due to the recent news surrounding airlines not staying in the air, the search term "flight" required some scrolling before I got to older articles of relevance regarding Time of Flight on Wikipedia.  

I learned that Time of flight is the measurement of the time taken by an object to travel a distance in a medium.  This is important, because the information can be used to establish a time standard to measure velocity or distance (read: depth).

How does a time-of-flight camera work?

When using a time-of-flight camera, such as what's coming out on the Huawei P30, the camera will illuminate the subject area with a modulated light source.  The sensor then receives the reflected light off objects, and can calculate distance between the source and subject.

Image captured by a 3D-TOF-camera (Wikipedia)

What is the application?

While having this on a phone may seem interesting at first, the practical application may be a few years lagging.  Immediately, I could think of a few uses, where you are able to capture "3D images" of an area for potential augmentation in a game, 3D imaging applications, or modeling applications.  It would be a great merging of augmented reality currently seen in games (think Pokemon), and a more grown up use of the mainstream technology, into a practical and usable tech that everyone would benefit from.

There are a ton of other applications for the time of flight sensors, and many have been used for quite some time.  I was honestly surprised to see all the practical applications for the sensor outside of 3D images and cameras.

Do I think I'll see the time of flight sensors coming to more phones anytime soon? Honestly, that's a hard call to make.  I do see a real use case outside of trippy selfies, but I am not really sure that the audience is big enough.

The problem is Huawei has this sensor, and isn't allowed into the huge United States market.  On March 7, 2019, Huawei announced that it filed a complaint with the U.S. federal court.  That complaint challenges the constitutionality of Section 889 of the 2019 National Defense Authorization Act.

"The U.S. Congress has repeatedly failed to produce any evidence to support its restrictions on Huawei products. We are compelled to take this legal action as a proper and last resort," Guo Ping, Huawei Rotating Chairman said. "This ban not only is unlawful, but also restricts Huawei from engaging in fair competition, ultimately harming U.S. consumers. We look forward to the court's verdict, and trust that it will benefit both Huawei and the American people."  (Huawei press release)

If successful, we should see the Huawei products back in the US market.  After the US market is allowed to play with the time of flight cameras, we may see the ultimate decision to keep or remove the camera based on that reaction.

For those interested in the actual section of the NDAA that Huawei makes reference to, this is what it looks like:

SEC. 889. PROHIBITION ON CERTAIN TELECOMMUNICATIONS AND VIDEO SURVEILLANCE SERVICES OR EQUIPMENT. (a) PROHIBITION ON USE OR PROCUREMENT.—(1) The head of an executive agency may not— (A) procure or obtain or extend or renew a contract to procure or obtain any equipment, system, or service that uses covered telecommunications equipment or services as a substantial or essential component of any system, or as critical technology as part of any system; or (B) enter into a contract (or extend or renew a contract) with an entity that uses any equipment, system, or service that uses covered telecommunications equipment or services as a substantial or essential component of any system, or as critical technology as part of any system. (2) Nothing in paragraph (1) shall be construed to— (A) prohibit the head of an executive agency from procuring with an entity to provide a service that connects to the facilities of a third-party, such as backhaul, roaming, or interconnection arrangements; or (B) cover telecommunications equipment that cannot route or redirect user data traffic or permit visibility into any user data or packets that such equipment transmits or otherwise handles. (b) PROHIBITION ON LOAN AND GRANT FUNDS.—(1) The head of an executive agency may not obligate or expend loan or grant funds to procure or obtain, extend or renew a contract to procure or obtain, or enter into a contract (or extend or renew a contract) to procure or obtain the equipment, services, or systems described in subsection (a). (2) In implementing the prohibition in paragraph (1), heads of executive agencies administering loan, grant, or subsidy programs, including the heads of the Federal Communications Commission, the Department of Agriculture, the Department of Homeland Security, the Small Business Administration, and the Department of Commerce, shall prioritize available funding and technical support to assist affected businesses, institutions and organizations as is reasonably necessary for those affected entities to transition from covered communications equipment and services, to procure replacement equipment and services, and to ensure that communications service to users and customers is sustained. (3) Nothing in this subsection shall be construed to— (A) prohibit the head of an executive agency from procuring with an entity to provide a service that connects to the facilities of a third-party, such as backhaul, roaming, or interconnection arrangements; or (B) cover telecommunications equipment that cannot route or redirect user data traffic or permit visibility into any user data or packets that such equipment transmits or otherwise handles. (c) EFFECTIVE DATES.—The prohibition under subsection (a)(1)(A) shall take effect one year after the date of the enactment of this Act, and the prohibitions under subsections (a)(1)(B) and (b)(1) shall take effect two years after the date of the enactment of this Act. (d) WAIVER AUTHORITY.— H. R. 5515—283 (1) EXECUTIVE AGENCIES.—The head of an executive agency may, on a one-time basis, waive the requirements under subsection (a) with respect to an entity that requests such a waiver. The waiver may be provided, for a period of not more than two years after the effective dates described in subsection (c), if the entity seeking the waiver— (A) provides a compelling justification for the additional time to implement the requirements under such subsection, as determined by the head of the executive agency; and (B) submits to the head of the executive agency, who shall not later than 30 days thereafter submit to the appropriate congressional committees, a full and complete laydown of the presences of covered telecommunications or video surveillance equipment or services in the entity’s supply chain and a phase-out plan to eliminate such covered telecommunications or video surveillance equipment or services from the entity’s systems. (2) DIRECTOR OF NATIONAL INTELLIGENCE.—The Director of National Intelligence may provide a waiver on a date later than the effective dates described in subsection (c) if the Director determines the waiver is in the national security interests of the United States. (f) DEFINITIONS.—In this section: (1) APPROPRIATE CONGRESSIONAL COMMITTEES.—The term ‘‘appropriate congressional committees’ ’’ means— (A) the Committee on Banking, Housing, and Urban Affairs, the Committee on Foreign Relations, and the Committee on Homeland Security and Governmental Affairs of the Senate; and (B) the Committee on Financial Services, the Committee on Foreign Affairs, and the Committee on Oversight and Government Reform of the House of Representatives. (2) COVERED FOREIGN COUNTRY.—The term ‘‘covered foreign country’’ means the People’s Republic of China. (3) COVERED TELECOMMUNICATIONS EQUIPMENT OR SERVICES.—The term ‘‘covered telecommunications equipment or services’’ means any of the following: (A) Telecommunications equipment produced by Huawei Technologies Company or ZTE Corporation (or any subsidiary or affiliate of such entities). (B) For the purpose of public safety, security of government facilities, physical security surveillance of critical infrastructure, and other national security purposes, video surveillance and telecommunications equipment produced by Hytera Communications Corporation, Hangzhou Hikvision Digital Technology Company, or Dahua Technology Company (or any subsidiary or affiliate of such entities). (C) Telecommunications or video surveillance services provided by such entities or using such equipment. (D) Telecommunications or video surveillance equipment or services produced or provided by an entity that the Secretary of Defense, in consultation with the Director of the National Intelligence or the Director of the Federal Bureau of Investigation, reasonably believes to be an entity owned or controlled by, or otherwise connected to, the government of a covered foreign country. H. R. 5515—284 (4) EXECUTIVE AGENCY.—The term ‘‘executive agency’’ has the meaning given the term in section 133 of title 41, United States Code. (congress.gov)