|Exam Name||:||IBM Optimization Technical Mastery Test v1|
|Questions and Answers||:||30 Q & A|
|Updated On||:||February 15, 2019|
|PDF Download Mirror||:||Pass4sure 000-N07 Dump|
|Get Full Version||:||Pass4sure 000-N07 Full Version|
Where can I download 000-N07 latest dumps?
The killexams.com dumps provide the study dump with the right abilties. Their Dumps are making learning smooth and brief to prepare. The provided dump is particularly custom designed with out turning intooverwhelming or burdensome. The ILT ebook is used in conjunction with their material and discovered its effectiveness. I recommendthis to my pals on the place of work and to all of us looking for the high-quality answer for the 000-N07 exam. Thank you.
Get those 000-N07 Q&A, put together and chillout!
I prepare people for 000-N07 exam challenge and refer all for your site for in addition advanced getting ready. This is definitely the high-quality site that offers strong exam material. This is the excellent asset I realize of, as I had been going to numerous locales if no longer all, and I have presumed that killexams.com Dumps for 000-N07 is definitely up to the mark. Much obliged killexams.com and the exam simulator.
where am i capable of find take a look at guide for actual knowledge brand new 000-N07 exam?
My parents told me their stories that they used to study very seriously and passed their exam in first attempt and our parents never bothered about our education and career building. With due respect I would like to ask them that were they taking the 000-N07 exam and confronted with the flood of books and study guides that confuse students during their exam studies. Definitely the answer will be NO. But today you cannot run off from these certifications through 000-N07 exam even after completing your conventional education and then what to talk of a career building. The prevailing competition is cut-throat. However, you do not have to worry because killexams.com questions and answers are there which is fair enough to take the students to the point of exam with confidence and assurance of passing 000-N07 exam. Thanks a lot to killexams.com team otherwise we shall be scolding by our parents and listening their success stories.
Unbelieveable! but authentic source modern-day 000-N07 real test questions.
before discovering this high-quality killexams.com, i used to be genuinely certain approximately competencies of the net. as soon as I made an account right here I noticed a whole new international and that become the beginning of my successful streak. so that you can get absolutely organized for my 000-N07 test, i was given a number of examine questions / answers and a set sample to follow which became very precise and comprehensive. This assisted me in achieving achievement in my 000-N07 test which become an super feat. thank you plenty for that.
Can i get ultra-modern dumps with actual Q & A ultra-modern 000-N07 examination?
In case you want excessive satisfactory 000-N07 dumps, then killexams.com is the final desire and your best solution. It givesincredible and notable check dumps which i am saying with full self warranty. I normally notion that 000-N07 dumps are of no uses however killexams.com proved me wrong due to the fact the dumps supplied by using them had been of excellent use and helped me marks excessive. In case you are demanding for 000-N07 dumps as nicely, you then need not to worry and be part of killexams.
Very clean to get licensed in 000-N07 exam with these Q&A.
I have never used the sort of super Dumps for my mastering. It assisted rightly for the 000-N07 exam. I already used the killexams.com killexams.com and passed my 000-N07 exam. It is the flexible material to apply. However, I changed into a underneath common candidate, it made me pass inside the exam too. I used best killexams.com for the studying and by no means used another material. I will preserve on using your product for my future tests too. Got ninety eight%.
What do you mean with the aid of 000-N07 exam?
I handed the 000-N07 certification nowadays with the help of your supplied Questions answers. This blended with the path that you need to take that allows you to grow to be a certified is the manner to move. In case you do but suppose that simply remembering the questions and answers is all you need to pass nicely you are incorrect. There had been pretty some questions aboutthe exam that arent inside the provided QA however if you prepare numerous these Questions answers; you could strive those very easily. Jack from England
What do you mean with the aid of 000-N07 exam?
Going through killexams.com Q&A has turn out to be a addiction while exam 000-N07 comes. And with tests developing in pretty a lot 6 days Q&A was getting greater critical. However with topics I need some reference guide to move from time to time so that i would get better assist. Way to killexams.com their Q&A that made all of it easy to get the topics internal your head effortlessly which may otherwise will be not possible. And its miles all due to killexams.com merchandise that I managed to score 980 in my exam. Thats the very satisfactory score in my class.
Do a quick and smart move, prepare these 000-N07 Questions and Answers.
a few rightly men cant bring an alteration to the worlds way however they can most effective inform you whether you have got been the simplest man who knew how to do that and i want to be acknowledged on this world and make my personal mark and i have been so lame my complete way but I realize now that I wanted to get a pass in my 000-N07 and this could make me well-known perhaps and yes im quick of glory however passing my A+ tests with killexams.com changed into my morning and night glory.
Did you attempted this top notch supply modern-day dumps.
Before discovering this great killexams.com, i used to be without a doubt effective approximately capabilities of the net. As soon as I made an account here I observed a whole new worldwide and that was the beginning of my successful streak. That lets in you toget definitely prepared for my 000-N07 checks, i used to be given quite a few test questions / answers and a difficult and fastpattern to test which became very precise and entire. This assisted me in conducting fulfillment in my 000-N07 test which end up an excellent feat. Thanks loads for that.
Over on the IBM blog, IBM Fellow Hillary Hunter writes that the enterprise anticipates that the area’s quantity of digital records will exceed 44 zettabytes, an excellent quantity. As organisations start to realize the sizeable, untapped potential of information, they need to discover a way to exploit it. Enter AI.
IBM has worked to construct the business’s most comprehensive facts science platform. built-in with NVIDIA GPUs and software designed specially for AI and the most data-intensive workloads, IBM has infused AI into offerings that customers can access regardless of their deployment model. these days, we take the subsequent step in that adventure in announcing the next evolution of our collaboration with NVIDIA. We plan to leverage their new information science toolkit, RAPIDS, across our portfolio in order that our valued clientele can increase the performance of computing device gaining knowledge of and information analytics.
Plans to promote GPU-accelerated desktop researching include:
IBM and NVIDIA’s shut collaboration through the years has helped leading enterprises and organizations all over the world tackle one of the most world’s largest complications,” talked about Ian Buck, vice president and prevalent supervisor of Accelerated Computing at NVIDIA. “Now, with IBM taking expertise of RAPIDS open-supply libraries announced nowadays by means of NVIDIA, GPU accelerated computer gaining knowledge of is coming to statistics scientists, assisting them analyze huge information for insights sooner than ever possible earlier than. Recognizing the computing vigor that AI would want, IBM changed into an early suggest of data-centric programs. This approach led us to bring the GPU-outfitted Summit device, the world’s strongest supercomputer, and already researchers are seeing gigantic returns. prior in the yr, we validated the advantage for GPUs to speed up machine studying once we confirmed how GPU-accelerated computer learning on IBM energy techniques AC922 servers set a new velocity record with a 46x development over outdated effects.
because of IBM’s commitment to bringing accelerated AI to users throughout the know-how spectrum, be they users of on-premises, public cloud, private cloud, or hybrid cloud environments, the enterprise is placed to deliver RAPIDS to users despite how they wish to entry them.
Hillery Hunter is an IBM Fellow and CTO of Infrastructure within the IBM Hybrid Cloud enterprise. earlier than this role, she served as Director of Accelerated Cognitive Infrastructure in IBM analysis, main a team doing move-stack (hardware via software) optimization of AI workloads, producing productivity breakthroughs of 40x and better which were transferred into IBM product choices. Her technical interests have all the time been interdisciplinary, spanning from silicon expertise through device utility, and she or he has served in technical and leadership roles in memory know-how, techniques for AI, and different areas. She is a member of the IBM Academy of expertise.
sign in for our insideHPC publication
Feb 08, 2019 (Heraldkeeper by means of COMTEX) -- a new research doc is delivered in HTF MI database of 200 pages, titled as ‘world IT Sourcing Market dimension study, via functions (software construction, net building, utility assist and administration, aid Desk, Database development and administration, Telecommunication), with the aid of conclusion clients (executive, BFSI, Telecom, Others), and Regional Forecasts 2018-2025′ with exact analysis, competitive panorama, forecast and methods. The look at covers geographic analysis that comprises regions like North the us, South the us, Asia, Europe & Others and significant players/providers such as Accenture, IBM corporation, Cisco systems, CA applied sciences, HP employer, excellent systems, Synnex corporation, Dell applied sciences. The record will help you profit market insights, future traits and boom prospects for forecast duration of 2018 - 2025.
Request a sample file @ https://www.htfmarketreport.com/pattern-document/1623525-global-it-sourcing-market-measurement-examine-by using-functions
global IT Sourcing Market valued about USD xxx million in 2017 is predicted to grow with a suit increase fee of greater than xxx% over the forecast length 2018-2025. The IT Sourcing is developing and expanding at a significant pace. The information expertise (IT) outsourcing is precisely referred to the sub-contracting of certain features or to pursue substances backyard an enterprise for all or a person a part of an IT function which would not have an awful lot of technical potential. The short-time period advice or the more cost-effective charges on elementary task are the main reasons why groups operating within the current state of affairs outsource work. The Outsourcing method allows staffing flexibility for an business together with allows them to bring in further components as and when required & extra unlock them when they are executed, therefore gratifying the cyclic or seasonal demand. The IT outsourcing market is essentially driven due to escalating should optimize enterprise procedures, surging integration of software outsourcing and potential optimization considering that the international state of affairs.
Get Customization in the record, Enquire Now @ https://www.htfmarketreport.com/enquiry-earlier than-purchase/1623525-international-it-sourcing-market-dimension-study-by way of-functions
The main market avid gamers exceptionally include-AccentureIBM CorporationCisco SystemsCA TechnologiesHP CorporationQuality SystemsSynnex CorporationDell applied sciences
The objective of the study is to outline market sizes of distinctive segments & international locations in fresh years and to forecast the values to the coming eight years. The document is designed to include each qualitative and quantitative facets of the business within each and every of the regions and countries concerned within the look at. moreover, the file also caters the specified assistance concerning the critical facets similar to using components & challenges with the intention to outline the longer term growth of the market. moreover, the report shall also include available opportunities in micro markets for stakeholders to make investments together with the specified analysis of aggressive landscape and product offerings of key avid gamers. The detailed segments and sub-section of the market are defined under:
by way of functions:software DevelopmentWeb DevelopmentApplication help and ManagementHelp DeskDatabase development and ManagementTelecommunication
by using end clients:GovernmentBFSITelecomOthers
by regions:North AmericaEuropeAsia PacificLatin AmericaRest of the world
in addition, years considered for the study are as follows:
historical 12 months - 2015, 2016Base 12 months - 2017Forecast duration - 2018 to 2025
goal viewers of the international IT Sourcing Market in Market examine:Key Consulting agencies & AdvisorsLarge, medium-sized, and small enterprisesVenture capitalistsValue-added Resellers (VARs)Third-birthday party talents providersInvestment bankersInvestors
buy this record @ https://www.htfmarketreport.com/buy-now?layout=1&report=1623525
table OF CONTENTSChapter 1. international IT Sourcing Market Definition and Scope1.1. research Objective1.2. Market Definition1.3. Scope of The Study1.4. Years considered for The Study1.5. currency Conversion Rates1.6. record LimitationChapter 2. analysis Methodology2.1. research Process2.1.1. information Mining2.1.2. Analysis2.1.3. Market Estimation2.1.four. Validation2.1.5. Publishing2.2. research AssumptionChapter three. govt Summary3.1. international & Segmental Market Estimates & Forecasts, 2015-2025 (USD Billion)3.2. Key TrendsChapter four. international IT Sourcing Market Dynamics4.1. increase Prospects4.1.1. Drivers4.1.2. Restraints4.1.three. Opportunities4.2. business Analysis4.2.1. Porter's 5 force Model4.2.2. PEST Analysis4.2.3. value Chain Analysis4.three. Analyst suggestion & ConclusionChapter 5. international IT Sourcing Market, via Services5.1. Market Snapshot5.2. Market performance – expertise Model5.three. world IT Sourcing Market, Sub section Analysis5.three.1. application Development5.three.1.1. Market estimates & forecasts, 2015-2025 (USD Billion)….persisted
View distinctive desk of content @ https://www.htfmarketreport.com/reviews/1623525-global-it-sourcing-market-measurement-analyze-through-functions
It’s a must-have you hold your market potential up so far. if you have a distinct set of avid gamers/manufacturers in line with geography or wants regional or country segmented experiences we can deliver customization hence.
Unquestionably it is hard assignment to pick dependable certification questions/answers assets regarding review, reputation and validity since individuals get sham because of picking incorrectly benefit. Killexams.com ensure to serve its customers best to its assets concerning exam dumps update and validity. The vast majority of other's sham report dissension customers come to us for the brain dumps and pass their exams joyfully and effortlessly. We never trade off on our review, reputation and quality on the grounds that killexams review, killexams reputation and killexams customer certainty is imperative to us. Uniquely we deal with killexams.com review, killexams.com reputation, killexams.com sham report objection, killexams.com trust, killexams.com validity, killexams.com report and killexams.com scam. On the off chance that you see any false report posted by our rivals with the name killexams sham report grievance web, killexams.com sham report, killexams.com scam, killexams.com protest or something like this, simply remember there are constantly awful individuals harming reputation of good administrations because of their advantages. There are a huge number of fulfilled clients that pass their exams utilizing killexams.com brain dumps, killexams PDF questions, killexams hone questions, killexams exam simulator. Visit Killexams.com, our specimen questions and test brain dumps, our exam simulator and you will realize that killexams.com is the best brain dumps site.
1Z0-071 dumps | HP0-S12 questions and answers | 1Z0-439 VCE | ADM-201 braindumps | 2V0-621D exam questions | TB0-111 pdf download | C2090-735 dump | PPM-001 practice questions | 000-189 practice questions | LOT-805 study guide | STI-884 cheat sheets | 000-N02 braindumps | 920-141 real questions | LOT-822 sample test | C2020-930 dumps questions | HP0-M14 braindumps | VCPN610 practice exam | E22-214 practice test | 1Y0-900 free pdf | 310-102 questions answers |
Pass4sure 000-N07 Practice Tests with Real Questions
Simply experience our Questions bank and feel certain about the 000-N07 test. You will pass your exam at high marks or your cash back. We have collected a database of 000-N07 Dumps from real exams to allow you to prepare and pass 000-N07 exam on the specific first attempt. Basically set up our Exam Simulator and prepare. You will pass the exam.
If you are attempting to find Pass4sure IBM 000-N07 Dumps containing actual exams questions and answers for the IBM Optimization Technical Mastery Test v1 Exam instruction, we provide most up to date and quality wellspring of 000-N07 Dumps this is http://killexams.com/pass4sure/exam-detail/000-N07. We have aggregated a database of 000-N07 Dumps questions from real exams with a selected cease purpose to give you a risk free get ready and pass 000-N07 exam at the first attempt.
killexams.com Huge Discount Coupons and Promo Codes are as below;
WC2017 : 60% Discount Coupon for all tests on website
PROF17 : 10% Discount Coupon for Orders more than $69
DEAL17 : 15% Discount Coupon for Orders extra than $ninety nine
DECSPECIAL : 10% Special Discount Coupon for All Orders
At killexams.com, we provide thoroughly reviewed IBM 000-N07 training resources which are the best for Passing 000-N07 test, and to get certified by IBM. It is a best choice to accelerate your career as a professional in the Information Technology industry. We are proud of our reputation of helping people pass the 000-N07 test in their very first attempts. Our success rates in the past two years have been absolutely impressive, thanks to our happy customers who are now able to boost their career in the fast lane. killexams.com is the number one choice among IT professionals, especially the ones who are looking to climb up the hierarchy levels faster in their respective organizations. IBM is the industry leader in information technology, and getting certified by them is a guaranteed way to succeed with IT careers. We help you do exactly that with our high quality IBM 000-N07 training materials.
IBM 000-N07 is omnipresent all around the world, and the business and software solutions provided by them are being embraced by almost all the companies. They have helped in driving thousands of companies on the sure-shot path of success. Comprehensive knowledge of IBM products are required to certify a very important qualification, and the professionals certified by them are highly valued in all organizations.
We provide real 000-N07 pdf exam questions and answers braindumps in two formats. Download PDF & Practice Tests. Pass IBM 000-N07 real Exam quickly & easily. The 000-N07 braindumps PDF type is available for reading and printing. You can print more and practice many times. Our pass rate is high to 98.9% and the similarity percentage between our 000-N07 study guide and real exam is 90% based on our seven-year educating experience. Do you want achievements in the 000-N07 exam in just one try?
Cause all that matters here is passing the 000-N07 - IBM Optimization Technical Mastery Test v1 exam. As all that you need is a high score of IBM 000-N07 exam. The only one thing you need to do is downloading braindumps of 000-N07 exam study guides now. We will not let you down with our money-back guarantee. The professionals also keep pace with the most up-to-date exam in order to present with the the majority of updated materials. Three Months free access to be able to them through the date of buy. Every candidates may afford the 000-N07 exam dumps via killexams.com at a low price. Often there is a discount for anyone all.
In the presence of the authentic exam content of the brain dumps at killexams.com you can easily develop your niche. For the IT professionals, it is vital to enhance their skills according to their career requirement. We make it easy for our customers to take certification exam with the help of killexams.com verified and authentic exam material. For a bright future in the world of IT, our brain dumps are the best option.
killexams.com Huge Discount Coupons and Promo Codes are as under;
WC2017 : 60% Discount Coupon for all exams on website
PROF17 : 10% Discount Coupon for Orders greater than $69
DEAL17 : 15% Discount Coupon for Orders greater than $99
DECSPECIAL : 10% Special Discount Coupon for All Orders
A top dumps writing is a very important feature that makes it easy for you to take IBM certifications. But 000-N07 braindumps PDF offers convenience for candidates. The IT certification is quite a difficult task if one does not find proper guidance in the form of authentic resource material. Thus, we have authentic and updated content for the preparation of certification exam.
Killexams 70-523-CSharp cram | Killexams 9L0-608 dump | Killexams HP2-H18 practice exam | Killexams 922-098 bootcamp | Killexams CFE study guide | Killexams F50-526 study guide | Killexams HP2-K14 brain dumps | Killexams 70-698 Practice test | Killexams 030-333 questions and answers | Killexams 00M-220 practice test | Killexams 9A0-502 VCE | Killexams 050-707 practice test | Killexams 000-567 free pdf | Killexams M2090-743 braindumps | Killexams JN0-140 questions answers | Killexams 1Z0-133 dumps | Killexams HP2-056 free pdf | Killexams 050-80-CASECURID01 real questions | Killexams 70-554-VB free pdf download | Killexams 1Z1-591 braindumps |
Killexams HP0-409 pdf download | Killexams OG0-021 real questions | Killexams 050-894 study guide | Killexams 050-653 brain dumps | Killexams ES0-005 practice test | Killexams C2080-470 exam prep | Killexams 1T6-520 bootcamp | Killexams 000-271 questions and answers | Killexams HP0-W02 dumps | Killexams 000-M226 free pdf download | Killexams P2090-032 cheat sheets | Killexams 000-605 test prep | Killexams 70-685 real questions | Killexams A2040-924 free pdf | Killexams 000-539 mock exam | Killexams C2030-136 questions answers | Killexams VCS-255 exam questions | Killexams A2090-735 dumps questions | Killexams 250-370 free pdf | Killexams C2020-011 VCE |
Ricardo Balduino and Tim BohnEarly Flight, Creative Commons Introduction
As we described in Part 1 of this series, our objective is to help predict the probability of the cancellation of a flight between two of the ten U.S. airports most affected by weather conditions. We use historical flights data and historical weather data to make predictions for upcoming flights.
Over the course of this four-part series, we use different platforms to help us with those predictions. Here in Part 2, we use the IBM SPSS Modeler and APIs from The Weather Company.Tools used in this use case solution
IBM SPSS Modeler is designed to help discover patterns and trends in structured and unstructured data with an intuitive visual interface supported by advanced analytics. It provides a range of advanced algorithms and analysis techniques, including text analytics, entity analytics, decision management and optimization to deliver insights in near real-time. For this use case, we used SPSS Modeler 18.1 to create a visual representation of the solution, or in SPSS terms, a stream. That’s right — not one line of code was written in the making of this blog.
We also used The Weather Company APIs to retrieve historical weather data for the ten airports over the year 2016. IBM SPSS Modeler supports calling the weather APIs from within a stream. That is accomplished by adding extensions to SPSS, available in the IBM SPSS Predictive Analytics resources page, a.k.a. Extensions Hub.A proposed solution
In this blog, we propose one possible solution for this problem. It’s not meant to be the only or the best possible solution, or a production-level solution for that matter, but the discussion presented here covers the typical iterative process (described in the sections below) that helps us accumulate insights and refine the predictive model across iterations. We encourage the readers to try and come up with different solutions, and provide us with your feedback for future blogs.Business and data understanding
The first step of the iterative process includes understanding and gathering the data needed to train and test our model later.
Flights data — We gathered 2016 flights data from the US Bureau of Transportation Statistics website. The website allows us to export one month at a time, so we ended up with 12 csv (comma separated value) files. We used IBM SPSS Modeler to merge all the csv files into one set and to select the ten airports in our scope. Some data clean-up and formatting was done to validate dates and hours for each flight, as seen in Figure 1.Figure 1 — gathering and preparing flights data in IBM SPSS Modeler
Weather data — From the Extensions Hub, we added the TWCHistoricalGridded extension to SPSS Modeler, which made the extension available as a node in the tool. That node took a csv file listing the 10 airports latitude and longitude coordinates as input, and generated the historical hourly data for the entire year of 2016, for each airport location, as seen in Figure 2.Figure 2 — gathering and preparing weather data in IBM SPSS Modeler
Combined flights and weather data — To each flight in the first data set, we added two new columns: ORIGIN and DEST, containing the respective airport codes. Next, flight data and the weather data were merged together. Note: the “stars” or SPSS super nodes in Figure 3 are placeholders for the diagrams in Figures 1 and 2 above.Figure 3 — combining flights and weather data in IBM SPSS Modeler Data preparation, modeling, and evaluation
We iteratively performed the following steps until the desired model qualities were reached:
· Prepare data
· Perform modeling
· Evaluate the model
Figure 4 shows the first and second iterations of our process in IBM SPSS Modeler.Figure 4 — iterations: prepare data, run models, evaluate — and do it again First iteration
To start preparing the data, we used the combined flights and weather data from the previous step and performed some data cleanup (e.g. took care of null values). In order to better train the model later on, we filtered out rows where flight cancellations were not related to weather conditions (e.g. cancellations due to technical issues, security issues, etc.)Figure 5 — imbalanced data found in our input data set
This is an interesting use case, and often a hard one to solve, due to the imbalanced data it presents, as seen in Figure 5. By “imbalanced” we mean that there were far more non-cancelled flights in the historical data than cancelled ones. We will discuss how we dealt with imbalanced data in the following iteration.
Next, we defined which features were required as inputs to the model (such as flight date, hour, day of the week, origin and destination airport codes, and weather conditions), and which one was the target to be generated by the model (i.e. predict the cancellation status). We then partitioned the data into training and testing sets, using an 85/15 ratio.
The partitioned data was fed into an SPSS node called Auto Classifier. This node allowed us to run multiple models at once and preview their outputs, such as the area under the ROC curve, as seen in Figure 6.Figure 6 — models output provided by the Auto Classifier node
That was a useful step in making an initial selection of a model for further refinement during subsequent iterations. We decided to use the Random Trees model since the initial analysis showed it has the best area under the curve as compared to the other models in the list.Second iteration
During the second iteration, we addressed the skewedness of the original data. For that purpose, we chose one of the SPSS nodes called SMOTE (Synthetic Minority Over-sampling Technique). This node provides an advanced over-sampling algorithm that deals with imbalanced datasets, which helped our selected model work more effectively.Figure 7 — distribution of cancelled and non-cancelled flights after using SMOTE
In Figure 7, we notice a more balanced distribution between cancelled and non-cancelled flights after running the data through SMOTE.
As mentioned earlier, we picked the Random Trees model for this sample solution. This SPSS node provides a model for tree-based classification and prediction that is built on Classification and Regression Tree methodology. Due to its characteristics, this model is much less prone to overfitting, which gives a higher likelihood of repeating the same test results when you use new data, that is, data that was not part of the original training and testing data sets. Another advantage of this method — in particular for our use case — is its ability to handle imbalanced data.
Since in this use case we are dealing with classification analysis, we used two common ways to evaluate the performance of the model: confusion matrix and ROC curve. One of the outputs of running the Random Trees model in SPSS is the confusion matrix seen in Figure 8. The table shows the precision achieved by the model during training.Figure 8 — Confusion Matrix for cancelled vs. non-cancelled flights
In this case, the model’s precision was about 95% for predicting cancelled flights (true positives), and about 94% for predicting non-cancelled flights (true negatives). That means, the model was correct most of the time, but also made wrong predictions about 4–5% of the time (false negatives and false positives).
That was the precision given by the model using the training data set. This is also represented by the ROC curve on the left side of Figure 9. We can see, however, that the area under the curve for the training data set was better than the area under the curve for the testing data set (right side of Figure 9), which means that during testing, the model did not perform as well as during training (i.e. it presented a higher rate of errors, or higher rate of false negatives and false positives).Figure 9 — ROC curves for the training and testing data sets
Nevertheless, we decided that the results were still good for the purposes of our discussion in this blog, and we stopped our iterations here. We encourage readers to further refine this model or even to use other models that could solve this use case.Deploying the model
Finally, we deployed the model as a REST API that developers can call from their applications. For that, we created a “deployment branch” in the SPSS stream. Then, we used the IBM Watson Machine Learning service available on IBM Bluemix here. We imported the SPSS stream into the Bluemix service, which generated a scoring endpoint (or URL) that application developers can call. Developers can also call The Weather Company APIs directly from their application code to retrieve the forecast data for the next day, week, and so on, in order to pass the required data to the scoring endpoint and make the prediction.
A typical scoring endpoint provided by the Watson Machine Learning service would look like the URL shown below.
https://ibm-watson-ml.mybluemix.net/pm/v1/score/flights-cancellation?accesskey=<provided by WML service>
By passing the expected JSON body that includes the required inputs for scoring (such as the future flight data and forecast weather data), the scoring endpoint above returns if a given flight is likely to be cancelled or not. This is seen in Figure 10, which shows a call being made to the scoring endpoint — and its response — using an HTTP requester tool available in a web browser.Figure 10 — actual request URL, JSON body, and response from scoring endpoint
Notice in the JSON response above that the deployed model predicted this particular flight from Newark to Chicago would be 88.8% likely to be cancelled, based on forecast weather conditions.Conclusion
IBM SPSS Modeler is a powerful tool that helped us visually create a solution for this use case without writing a single line of code. We were able to follow an iterative process that helped us understand and prepare the data, then model and evaluate the solution, to finally deploy the model as an API for consumption by application developers.Resources
The IBM SPSS stream and data used as the basis for this blog are available on GitHub. There you can also find instructions on how to download IBM SPSS Modeler, get a key for The Weather Channel APIs, and much more.
Royalty-free I3C; CFET parasitic variation modeling; Intel funds analog IP generation.
The MIPI Alliance released MIPI I3C Basic v1.0, a subset of the MIPI I3C sensor interface specification that bundles 20 of the most commonly needed I3C features for developers and other standards organizations. The royalty-free specification includes backward compatibility with I2C, 12.5 MHz multi-drop bus that is over 12 times faster than I2C supports, in-band interrupts to allow slaves to notify masters of interrupts, dynamic address assignment, and standardized discovery.
Efinix will expand its product offering, adding a 200K logic element FPGA to its lineup with the Triton T200. The T200 targets AI-driven products, and its architecture has enough LEs, DSP blocks, and on-chip RAM to deliver 1 TOPS for CNN at INT8 precision and 5 TOPS for BNN, according to Efinix CEO Sammy Cheung. The company also released samples of its Trion T20 FPGA.
Faraday Technology released multi-protocol video interface IP on UMC 28nm HPC. The Multi-Protocol Video Interface IP solution supports both transmitter (TX) and receiver (RX). The transmitter allows for MIPI and CMOS-IO combo solutions for package cost reduction and flexibility, while the receiver combo PHY includes MIPI, LVDS, subLVDS, HiSPi, and CMOS-I/O to support a diversified range of interfaces to CMOS image sensors. Target applications include panel and sensor interfaces, projectors, MFP, DSC, surveillance, AR and VR, and AI.
Analog tool and IP maker Movellus closed a second round of funding from Intel Capital. Movellus’ technology automatically generates analog IPs using digital implementation tools and standard cells. The company will use the funds to expand its customer base and to increase its portfolio of PLLs, DLLs and LDOs for use in semiconductor and system designs at advanced process nodes.
Imec and Synopsys completed a comprehensive sub-3nm parasitic variation modeling and delay sensitivity study of complementary FET (CFET) architectures. The QuickCap NX 3D field solver was used by Synopsys R&D and imec research teams to model the parasitics for a variety of device architectures and to identify the most critical device dimensions and properties, which allowed for optimization of CFET devices for better power/performance trade-offs.
Credo utilized Moortec’s Temperature Sensor and Voltage Monitor IP to optimize performance and increase reliability in its latest generation of SerDes chips. Moortec’s PVT sensors are utilized in all Credo standard products which are being deployed on system OEM linecards and 100G per lambda optical modules. Credo cited ease of integration and reduced time-to-market and project risk.
Wave Computing selected Mentor’s Veloce Strato emulation platform for functional verification and validation of its latest Dataflow Processor Unit chip designs, which will be used in the company’s next-generation AI system. Wave cited capacity and scaling advantages, breadth of virtual use models, reliability, and determinism as behind the choice.
MaxLinear adopted Cadence’s Quantus and Tempus timing signoff tools in developing the MxL935xx Telluride device, a 400Gbps PAM4 SoC using 16FF process technology. MaxLinear estimated they got 2X faster multi-corner extraction runtimes versus single-corner runs and 3X faster timing signoff flow.
The European Processor Initiative selected Menta as its provider of eFPGA IP. The EPI, a collaboration of 23 partners including Atos, BMW, CEA, Infineon and ST, has the objective of co-designing, manufacturing and bringing to market a system that supports the high-performance computing requirements of exascale machines.Jesse Allen (all posts)Jesse Allen is the Knowledge Center administrator and a senior editor at Semiconductor Engineering.
New technologies in neuroscience generate reams of data at an exponentially increasing rate, spurring the design of very-large-scale data-mining initiatives. Several supranational ventures are contemplating the possibility of achieving, within the next decade(s), full simulation of the human brain.
I question here the scientific and strategic underpinnings of the runaway enthusiasm for industrial-scale projects at the interface between “wet” (biology) and “hard” (physics, microelectronics and computer science) sciences. Rather than presenting the achievements and hopes fueled by big-data–driven strategies—already covered in depth in special issues of leading journals—I focus on three major issues: (i) Is the industrialization of neuroscience the soundest way to achieve substantial progress in knowledge about the brain? (ii) Do we have a safe “roadmap,” based on a scientific consensus? (iii) Do these large-scale approaches guarantee that we will reach a better understanding of the brain?
This “opinion” paper emphasizes the contrast between the accelerating technological development and the relative lack of progress in conceptual and theoretical understanding in brain sciences. It underlines the risks of creating a scientific bubble driven by economic and political promises at the expense of more incremental approaches in fundamental research, based on a diversity of roadmaps and theory-driven hypotheses. I conclude that we need to identify current bottlenecks with appropriate accuracy and develop new interdisciplinary tools and strategies to tackle the complexity of brain and mind processes.Introduction
This essay explores how the big-data revolution has started to have an impact on brain sciences and assesses the dangers of letting technology-driven—rather than concept-driven—strategies shape the future industrialization of neuroscience through the rapid emergence of very-large-scale data-mining initiatives. Among recent supranational ventures, the EPFL-IBM consortium “Blue Brain” (1), the European consortium “The Human Brain Project” (HBP) (2), the U.S. consortia BRAIN (3, 4) and “The Human Connectome” (5), and the privately owned Allen Institute (6) all flirt with the possibility of achieving, within the next decades, the full simulation of the human brain (Box 1). Although big-data initiatives have started an impressive thrust in brain research, I question here their impact on how the brain sciences are evolving and highlight the necessity of developing alternative scientific strategies.Box 1 “Big data” projects in brain sciences: Websites China:
Brain Project: Basic neuroscience, brain diseases and brain-inspired computing in progress (147).
After briefly reviewing the current advances and hopes that new technologies bring within range of modern brain research, I raise the possibility that, at the same time, scientific conduct is undergoing a radical societal change (section 1). I outline the risks generated by the big-data revolution in brain sciences, discussing various conceptual bottlenecks (sections 2 to 5). I illustrate practical and theoretical limitations that brute-force strategies may encounter in simulating the full brain (sections 6 and 7). I suggest safeguards that should be kept in mind in the new societal context dominated by “economics of promises” (section 8), and conclude with a list of positive recommendations.1. Big-data initiatives: A worldwide change of scientific strategy in brain studies?
The prevailing consensus in neuroscience is that technology has revolutionized our approach in looking at brain structure and function in relation to behavior (7, 8), and in multiple ways:
1) at the technical level: by extending the power of techniques of circuit identification beyond that already reached by genetic or viral approaches, enabling high-throughput optical manipulation of large–neural ensemble activity with single-cell and single-spike resolution in vivo (9–12);
2) at the methodological level: by imposing new standards in experimentation and data acquisition in direct relation with behavior (13, 14);
3) at the data production level: by compiling genomic, structural, and functional databases, the size of which (measured in petabytes) is orders of magnitude larger than that of a complete mammalian genome (15);
4) at the level of analysis: by the application of methods of dimensionality reduction (16, 17) and of pattern-searching algorithms specialized for high-dimensional spaces (18), used previously in statistics, machine learning, and physics;
5) at the modeling level: by the overwhelming development of optimization and Bayesian predictive methods (19, 20) and deep learning approaches (21), made possible by the countless dimension of the data reservoir.
The impact of technical advances on brain research has become such that a major change in reference animal models used in neuroscience has occurred in less than 10 years: most state-of-the-art techniques favor the use of few experimental species [e.g., zebrafish, mouse, and marmoset among the vertebrates (22, 23)] and have already consigned to relative oblivion those used traditionally for functional electrophysiology and cognitive mapping (e.g., rat, cat, ferret, and macaque). Simultaneously, outstanding progress in noninvasive imaging techniques (24) such as diffusion tensor imaging (DTI), functional magnetic resonance imaging (fMRI), and ultra high-field MRI, paired with sophisticated neuro-cognitive paradigms (25, 26) and multivariate analysis methods (27, 28), now reaches spatial-scale resolution and temporal precision ranges (25, 27) closer to those used in invasive physiology in nonhuman mammals (29), making cross-species comparison, including humans, feasible in the near future.
Because bold scientific claims increase with technological prowess, the field has also raised its level of self-criticism. Despite major advances in optogenetic control of neural activity patterns (9, 11, 12), “interventionist” neuroscience is still required to show its efficiency in unraveling neural mechanisms causal to behavior (30). Methods must be developed to untangle multiple sources of shared or context-dependent correlations. At a more macroscopic level, localizationist interpretations in brain imaging recently came under scrutiny, both at the paradigmatic and preprocessing level, leading to more controlled definitions of reference or “null” statistics (31, 32). Still unsolved is the obvious difficulty of “putting all together” across scales, when comparing, for instance, neural responses and neurovascular coupling dynamics (33–37). These discrepancies need to be resolved, because they highlight the risks of betting on ill-chosen instrumentation-imposed observables.
The major risks go well beyond technological misuses or misinterpretations. The present trend prefigures a radical societal change in scientific conduct, where new directions in science are launched by new tools rather than by new concepts (38). Many leading scientists and funding agencies now share the view that “progress in science depends on new techniques, new discoveries and new ideas, probably in that order” (39). The pressure has become such that, to receive funding and eventually publish high-impact papers, scientists are often required to use mouse-specific state-of-the-art techniques, irrespective of their adequacy. To some degree, wishful thinking has replaced the conceptual drive behind experiments, as if using the fanciest tools and exploiting the power of numbers could bring about some epiphany.
Although industrialization in scientific methods and practice successfully prevailed in the human genome sequencing project [(40); but see (41, 42)], it is unlikely that a similar brute-force approach will guarantee major advances in understanding brain complexity. Conceptual guidance is required to make the best use of technological advances, regardless of their obvious benefits. “Technology is a useful servant but a dangerous master.” As pointed out by Florian Engert, “the essential ingredient that turns a useless map” or database “into an invaluable resource” remains “the experimental design employed to gather and analyze the underlying data, and ultimately the thought process, creativity, and ingenuity that went into this design” (43). At a more conceptual level, barrier-breaking innovation paradoxically stems more often from unpredictable “rupture” processes than industrialized approaches. In numerous cases, seminal findings in neuroscience were chance discoveries and daring interpretations. These go well beyond the technological limits of observations and, sometimes, provide the missing but consensual experimental evidence of prior conceptualization formulated centuries earlier. Better tools in hand are just not enough.2. Bottlenecks in large-scale search studies: Big-data is not knowledge
Provided adequate funding, “big” is easy to acquire and accumulate but hard to classify, interpret, and make sense of. The sea of biological data creates the illusion of knowing “more,” whereas we should rather acknowledge our profound underestimation of how “complex” the brain is. Big data in biology is not limited to acquisition of vast numbers of observables. It further requires selection criteria to evaluate their strategic value, and sophisticated handling to extract knowledge. Classically, in information science, one distinguishes four levels in the so-called DIKW pyramids (44), ranging from “data” to “information” to “knowledge” and “wisdom” (understanding). We are currently facing an overflow of data without definite strategies to convert it into knowledge and eventually reach a better comprehension of the living brain.
“The search for a unified theory…remains at a rudimentary stage for the brain sciences.”
The most common target in large-scale enterprises flourishing around brain sciences is the generation of biochemical or structural catalogs, most often “static,” taking the form of localizationist atlases in brain-imaging studies or structural inventories at the molecular, cellular, or network level. Of course, static “atlases” imply sophisticated visualization and are sold as tangible deliverables that can be easily understood in layman’s terms. Their use often leads to overinterpretation, when the brain is reduced to a charted globe divided into islands and continents (45–48). Many specialists are aware of the need of rescaling the applicability of instrumental methods and redefining the strict validity range of the conclusions derived from these atlases (49, 50).
Only 20 to 30 years ago, neuroanatomical and neurophysiological information was relatively scarce while understanding mind-related processes seemed within reach. Nowadays, we are drowning in a flood of information. Paradoxically all sense of global understanding is in acute danger of getting washed away. Each overcoming of technological barriers opens a Pandora’s box by revealing hidden variables, mechanisms, and nonlinearities, adding new levels of complexity. By reaching the microscopic-scale resolution, advanced technologies have unveiled a new world of diversity and randomness, which was not apparent in pioneer functional studies using spike rate readout or mesoscopic imaging of reduced sensitivity (51–53). This contrast between meso- and microscale functional architectures attests to the necessity of putting more effort in understanding the “regularization” impact of emergence laws—operating in a bottom-up way—across successive levels of integration (see sections 3 and 7). Observations made in parallel with different instruments (sensitive to various spatiotemporal scales) should be combined to build realistic biophysical models to reconcile the loosely related observables across integration levels. In particular, one needs to extract better predictive tools to understand the neural basis of activation processes revealed by brain imaging and find ways of comparing quantitatively state-of-the-art morphological tracing with DTI. Only then could one envision a comprehensive and compressed multiscale functional and structural data repository.
Another approach may be to seek advice from equivalent big-data enterprises in other disciplines such as astrophysics and elementary particle research. Both of these routinely generate petabytes of data. Although particle research does not necessarily conjure up the theoretical viewpoint that we are crucially missing, generations of physicists have been exploring the multiscale complexity of physical matter on the basis of ever-increasing big-data collections (see section 7). Presently, the major difference with brain science is that theorists in particle physics field are involved before—and not after—the hypothesis-driven data are collected. They actively participate in the definition of collective infrastructures and the design of one-of-a-kind equipment shared by the entire experimentalist collectivity. The recommendation made here is that biologists, who are new to this field, should learn from physicists. As such, the roadmap from data to knowledge could be mapped out in a much clearer fashion and the dead ends, where no one has a clear idea of what to do with all the data, would be far less likely.
To summarize, the trend toward increased measurement sensitivity and more microscopic scales carries its own paradox: A digitized ersatz of lower dimensionality will never account for the multiscale complexity of the full brain. We should adapt our strategic planning so that conceptual efforts grow in a way that is commensurate with technological development—and not follow it, as is presently the case.3. Bottlenecks in multilevel analysis: The Marr-Poggio conundrum
One of the advertised “blue sky” goals of big-data–driven initiatives is to establish the subcellular and cellular mechanisms causal to behavior through an exhaustive reductionist analysis. The best-known roadmap for dealing with brain complexity was formulated by David Marr some 35 years ago (54). One way to look at the proposed hierarchy of analysis levels (Fig. 1) is to progress from the global “functional and computational” level, through the intermediate “algorithmic” level down to the “substrate” or “implementation” level. The two higher levels, computational and algorithmic, can be considered as the most generic and abstract, independent of the biological trick used to implement them. Marr argued that whereas “algorithms and mechanisms are empirically more accessible, …the level of computational theory…is critically important from an information-processing point of view…[because]…the nature of the computations that underlie perception [and, by extension, cognition] depends more upon the computational problems that have to be solved than upon the particular hardware in which their solutions are implemented” (54). Marr was convinced that a purely reductionist strategy, decomposing the global process into its elementary subcomponents, was “genuinely dangerous.” Trying to understand the emergence of cognition from neuronal responses “is like trying to understand a bird’s flight by studying only feathers. It just cannot be done.’’ Marr’s main intuition was that it is much more difficult to infer from the neural implementation level what algorithm the brain is using (bottom-up) than to reach the algorithmic level from the study of the computational problem that it is trying to solve (top-down along the hierarchy). The bottom-up “emergence” process arising from the interaction of local low-level biological processes remains an open issue today. The way in which sensory neurophysiology has conferred to single-neuron firing the embodiment of high-level psychological properties that can only be sensibly ascribed to a whole behaving organism is a striking example of mereological fallacy (30, 55).Fig. 1 The hierarchy of analysis levels [inspired by David Marr (54)].
The three levels of Marr’s hierarchy illustrated are (from top to bottom) function and computation at the higher level (3), algorithm at the intermediate level (2), and biophysical substrate at the lower level (1). Reductionist approaches progress from levels 3 to 1, whereas constructionism goes the opposite way, from 1 to 3. Two examples of the three-level analysis are given for two different biological processes: action potential (middle column) and synaptic plasticity (right column). The two upper levels of Marr’s hierarchy define the field of computational neuroscience (red inset), the scope of which is to identify generic computations and functions and their underlying algorithms, independently of the biophysical substrate of the process under study.
Despite the wealth of produced data, constructionist approaches are thus likely to produce mimicry by a brain ersatz, because of the difficulties of reverse inference (in this case, inferring function and behavior from neural-level activation). This prediction was recently computationally explored, by designing arbitrary experiments on an artificial brain-like artifact, a single microprocessor, to see if popular data analysis methods from neuroscience could elucidate the way in which it processes information and controls behavior (in the present case, three classic videogames) (56). Although the processor’s algorithmic flowchart was known a priori, classical interventionist neuroscience methods failed to explain how the processor works, regardless of the amount of data (30).
…bottom-up “emergence”…remains an open issue today.
The critical point remains that causal-mechanistic explanations are qualitatively different from understanding how a combination of component modules performing the computations at a lower level produces emergent behavior at a higher level.
The first difficulty arises because higher-level concepts are needed to understand the neural implementation level. So, even when causality is demonstrated, it makes sense only when all levels are considered together simultaneously: “Ion channels do not beat, heart cells do. Neural circuits do not feel pain, whole organisms do” (30). Some key studies illustrate the necessity of binding different levels in the experimental design itself—for instance, by linking the neural level with the theoretical context derived from preexisting behavioral knowledge. The supervised learning experiments engineered in single neurons recorded in visual cortex in vivo (57), for example, were conceived as the direct neural implementation (substrate level) of a hypothetical plasticity rule (58) (algorithmic level) derived from associative memory (59) and Ising (60) models (computational level).
A second difficulty comes from Marr’s “multiple realizability” argument, which states that the same function can be achieved through any number of different substrates (30, 54, 61). The impossibility of mapping behavior or function in a unequivocal way on the parametric state of the synaptic or conductance ensemble (defining observed dynamics of the neural net under study) was reproduced in simulation models of Aplysia (62, 63) and vertebrate cerebellum (64). This conundrum reveals unexpected complexity whichever way the hierarchy is read, from the computation or macro level to the substrate or micro level, or the reverse.
An additional hidden twist is that the biological substrate level may consist of nested sublevels, each operating at different biophysical scales. Tomaso Poggio emphasized how knowledge of the more elementary steps of information processing is required to account for the complexity of more global computations (65). The key issue is to determine the minimal stratification level needed to preserve the nonlinearities and self-organizing properties at higher integrative levels (66).
Refined electrophysiological studies in the early visual system show clear cases where most spiking-net models—by not giving enough descriptive depth to the biophysical substrate—are too simplified to self-generate low-level feature specificity (orientation selectivity, contrast invariance., and so forth): (i) Rather than the simplified +/− algebra of McCulloch-Pitts neurons, synaptic biophysics in vivo suggests a much richer algebra that includes scaling and division of excitatory inputs by inhibitory ones, where a digital “zero” in the target neuron output could mean either absence of incoming signal (what spiking nets generally assume) or the division or “veto” of an excitatory input by a strong concomitant shunting inhibition (66, 67). (ii) Although orientation selectivity is a hallmark of mammalian cortical organization, this feature selectivity is, in most spiking models, forced in an ad hoc way, by prespecified wiring rules between thalamus and cortex. Only the orientation preference map appears to be treated as an emergent property resulting from horizontal connection plasticity (68). This oversimplification is challenged when viewed from the conductance level: Voltage-clamp measurements in vivo, even in layer 4, reveal an unexpected level of nonlinear interaction and diversity between excitatory and inhibitory conductances (67, 69–71), which, in V1 simple cells, are hardly detectable (72) or absent at the spiking level (73). The consequence is that the same functional receptive field type, “simple” or “complex,” may indeed be produced by multiple dynamic interaction patterns between excitation and inhibition (71, 74). This unexpected wiring diversity in the synaptic genesis of V1 receptive fields concurs with statistical predictions made by multilayered convolutional models (75). By oversimplifying synaptic integration biophysics and limiting simulations to the spike level, most computational models trivialize the emergence of “higher-order” properties through a purely feedforward cascade (76, 77) when the principal wiring feature of sensory neocortex is—by far—synaptic reverberation and amplification (66).
In view of the weight presently given to spike-based feedforward processing and deep learning, the reexamination of conductance-based versus spike-based computing and the role given to synaptic reentry both sound essential. Bottlenecks in multiscale modeling are rarely addressed in depth, and, although it is agreed that nobody has the definitive solution, this remains a serious blow for “constructionist” models of the brain. Alternative viewpoints should be developed.4. Bottlenecks in reverse engineering: Lessons learned from the invertebrates
One safe way to handle big-data sets in vertebrates is to avoid the pitfalls known from pioneering studies in paucineuronal networks. Comparative neuroscience offers multiple test studies: (i) small, genetically tractable animal models (78), such as Caenorhabditis elegans; (ii) functionally identified clusters of giant cells, in sensory-motor ganglions in Aplysia and crustaceans; and (iii) transparent zebrafish, making the online imaging of the whole connectome possible (79). This suggests access to “full brain” descriptions with the reconstruction of causal structuro-functional relations matching canonical neuronal states with species-specific behavioral repertoires (14, 80, 81).
Yet, even with such elementary invariant-like systems, interindividual variability cannot be ignored. A counterintuitive finding in C. elegans is that there is no such thing as “simplicity” despite the reduced connectome (302 neurons, 6963 synapses, 890 gap junctions), even at the earliest stage of sensory processing. Averaging neuronal responses of a single olfactory cell is deceptive, because the activation of the same neuron, depending on the context, may lead to several possible behavioral outcomes (82). The main predictive signal of the response is the internal state of the functional assembly in which the cell participates, at the exact time when external inputs become processed. Similar state dependencies in neuronal processing have just started to be explored in vertebrates (83, 84).
Partial understanding of the functional extent and multiscale impact of contextual processing has been obtained in classical studies in the lobster’s stomato-gastric ganglion (85). By releasing diffusible neuromodulators, specialized “orchestra conductor” neurons change the conductance repertoire of the other individual neurons and allow them to participate at distinct times in a diversity of functional subnetworks (“assembly reconfigurability”). This feature highlights the impossibility of separating intrinsic (conductance repertoire, genomic expression) from extrinsic (synapses) features. The diffusive nature of the modulatory process and its dependency on the internal mesoscopic state generated by the recurrent synaptic activity open a yet largely unexplored scale of complexity.
A straightforward lesson from invertebrates is that a purely “Lego”-like reconstruction approach—based on the full reconstruction of the brain’s connectome and the gene expression, electrical, and morphological determinants profile of the major classes of its neural components (86, 87)—may be doomed from the start. Despite similar evidence in vertebrates, some doubt remains as to whether the versatility of the excitability pattern and the dependency of conductance repertoire expression on past brain states (and modulators) are taken at face value in classifications and nomenclatures of supposedly invariant identity determinants (88). Thus, the dynamic complexity revealed in simpler organisms provides a powerful warning against the use of purely bottom-up constructivist large-scale studies in higher organisms.5. Bottlenecks in evolutionary leaps: Anthropocentrism from “mouse” to “man”
“Understanding the brain” is often read as understanding the “human” brain. This anthropomorphic bias reveals a loss of perspective regarding the essence of living systems: their diversity, their adaptability, and their dependence on evolutionary history. Losing track of this perspective is dangerous, because only broad comparisons offer the potential to distinguish general principles from unimportant implementation details. If paving the way toward “a general theory of the brain” is a worthy goal, as we believe it is, then it is essential to conceive comparative physiology strategies, which allow us to discriminate between species-specific “bags of tricks” and canonical computations shared by living brains (30, 66, 89–92). Certain forms of computation and algorithms seem to be preserved (i.e., gain control, normalization, exponentiation, association, and coincidence detection), but the detailed mechanistic implementations are often species-specific and structure-dependent (30). Industrial-scale efforts are, by their present design, focused on limited behaviors and species, and thus orthogonal to a broad-enough perspective.
A second problem is that the human brain is probably among the most complex of nervous systems. This has led, without much strategic planning other than exploiting the availability of a genetically modifiable mammalian system, to the increasing use of the mouse as a model. Because it is a mammal, it must be similar to the human. Although the mouse model has produced important advances in the study of basic sensory-motor integration principles, it may be less appropriate for studying perceptual processes for modalities (vision) less adapted to its behavioral repertoire and, more obviously still, for higher cognitive functions. This is particularly true in species such as humans and other primates where sensory cortical processing involves elaborate reciprocal connectivity patterns linking sets of functionally distinct areas (93, 94), which are mostly absent in the mouse cortex.
A wiser alternative could be to refine approaches progressively and recursively according to species-specific behavior, and cognition repertory (95). Search for homologies should be validated on the basis of structural, functional, and cognitive similarities between species. The choice of the right species calls for increased efforts in comparative physiology, which have been downplayed since the start of the mouse dominance era. The choice of the right tasks requires new methods of behavior classification. By applying unsupervised learning methods to the largest possible set of coregistered neural data and behavioral observations, one may hope to achieve substantial dimensionality reduction and obtain an objective mapping of possible behavioral repertoires over a restricted ensemble of reproducible brain states, as has been done successfully in invertebrates (81).6. Simulating the brain: The cart before the horse—immaturity of paradigms and lack of hypothesis-driven design
A fundamental issue for large database generalization and validation is to provide universal paradigm or task standards that are optimized for the study of specific cognitive functions. For illustration’s sake, let us concentrate on an apparently “simple” case study, i.e., how to characterize neural processes involved in low-level visual perception.
In the search for generic sensory integration principles, how can we conceive a “good” stimulus set before we know what the system under study is designed to perceive (96)? The process cannot be formulated without priors, often linked with behavioral observations and hypothesis testing, and should probably be automated only after a progressive, informed, recursive, maybe even “old-fashioned,” phase of investigation. Presenting the largest spectrum of input statistics seems the appropriate way to push the sensory system to its information capacity limits (97) and explore the dependency of the neural code on external input statistics (70, 74, 98, 99). However, in practice, the battery of stimuli used to build large data sets faces unacknowledged technical constraints: Stimulus choices are often guided by the efficiency with which strong firing can be evoked—leading to a prevalence of high firing rates, more easily detectable by calcium fluorescence changes—rather than by information theory concepts (rate code/dense spiking versus spike-timing code/sparseness). The cognitive repertoire should also be used more carefully to constrain the choice of species: There is something odd in applying in the mouse, a nearly blind animal (100), a battery of stimulation paradigms based on decades of work on highly visual species (cat, macaque, and human) without paying attention to ethological differences in the reliance on vision [but see (101)]. Indeed, visual cortex may play different roles in different species; for instance, space coding during navigation—in concert with hippocampus—in rodents, versus primal perceptual sketch elaboration and form or motion extraction—in concert with higher cortical areas—in more visual species. Consequently, testing the responses of mouse primary visual cortex (V1) to a high-contrast classic Hollywood black-and-white movie (102) seems as inappropriate as studying pangolin olfaction with plumes of warm Parisian croissants. Conversely, searching for place or grid cells may be deceiving in nonhuman primate visual cortex when it makes sense in the rodent.
Choosing the right stimulus and species is not the only issue. Since the shift over the past 20 years from the anesthetized-paralyzed preparation to the behaving animal, the standardization of the global context has become a major concern (103). Visual responsiveness in the awake mouse depends heavily on locomotion and full-body action (83), rendering inseparable the sensory and motor components. However, a similar conditional dependency of visual processing has not been confirmed in higher mammals, where primary sensory and motor cortices are much less—or even not at all in the adult—directly interconnected. Consequently, the generalized use of “running-on-a-ball” paradigms in the rodent may have set a new behavioral standard for studying sensory responses, optimized to increase neural excitability in the rodent only, but reducing the global relevance to vision per se (66).
“Industrial-scale efforts are…orthogonal to a broad-enough perspective.”
The overall consequence is that, by imposing such artificial paradigms as the “standard tests” for brain observatories, each resulting data set will yield predictions restricted to specific contexts, but largely unrelated to “natural” behavior. Big-data initiatives in early vision have not yet put enough effort into defining parameters critical of the “naturalness” of the evoked sensory drive. As summarized by Bruno Olshausen, “the problem is not just that we lack the proper data, but that we do not even have the right conceptual framework for thinking about what is happening” (104). Similarly, however impressive they may be, all-optical “interventionist” paradigms do not signal the end of the quest: New conceptual frameworks are needed that “provide the mapping between large-scale neural data and behavior in an algorithmic sense and not just a correlative or even causal way” (30). The practical message here is that both paradigms and context—in which data are acquired—should be rationalized and justified on purely theoretical grounds, before becoming the norm of the industrialization stage.7. Simulating the brain—The cart without a driver: Missing a strong brain theory
Do we have a clear view of what can be expected from reverse engineering and embodied constructionism? Some of the large-scale initiatives recapitulate earlier constructionist approaches that tried to simulate brain circuits by building models “that are very closely linked to the detailed anatomical and physiological structure” of the brain, in hopes of “generating unanticipated functional insights based on emergent properties of neuronal structure.” The first attempts in the 1990s (105–107) were limited by the lack of prediction of rich enough behavioral repertoires and cognitive functions (108). Conversely, more engineering-oriented and simplified blackbox simulations (109) were criticized for their lack of descriptive depth (110). Even so, some success has been obtained by clever built-in top-down constraints. High-performance computing may change the odds (111), and experts agree that large-scale simulation should provide possible breakthroughs in system identification as has been the case for deep learning (112). Nevertheless, given the analytic intractability of the brain, the challenge of “putting all together” remains wide open. The major obstacle remains the lack of unifying theory and the relative paucity of top-down guidance by high-level knowledge derived from psychological studies of the mind.
In this section, I will review three correlated issues: (i) Are there theoretical conjectures indicating that a full spike-based brain simulation is not a realistic target? (ii) How do system and computational neurosciences integrate theory so far? and (iii) Are there alternative roadmaps to readdress what may be considered as an ill-posed problem?
Point 1: Because of their dominant bottom-up drive, the danger of the large-scale neuroscience initiatives is to produce purely descriptive ersatz of brain, sharing some of the internal statistics of the biological counterpart, at best the two first-order moments (mean and variance), but devoid of self-generated cognitive abilities. The numbers will certainly look right, but there is no guarantee that such simulated brains will work. This intuition resonates with theoretical conjectures based on pure logic. As early as the 1980s, a gedanken experiment was proposed by von der Malsburg which considered two brain-like assemblies, built with the exact same connectivity graph and producing the exact same averaged firing patterns. What would happen if a jitter of a few milliseconds was applied in the arrival time of each occurring spike (while keeping mean rate invariant)? Is there a critical jitter value that should not be exceeded, to keep alive the emergent properties of the graph (113, 114)? The same conjecture could be generalized at the second-order statistics level. Let us imagine that big data makes it possible to build a cortex-like digital machine where the variance of the distributions of synaptic weights afferent to (or efferent from) each neuron could be matched to those directly measured (over time) in the same ensembles of real synapses. Would one predict the mean and variance–equalized artificial network to be as operative as the real brain? Because—in real brains— the efficiency of individual synaptic weights and their spatial distribution are stabilized through associative plasticity and normalization processes (if our popular learning theories are right), plugging in simulated synapses mean and variance levels devoid of information content would result in an “averaged connectome” without memory of its past interactions with the outside world. Thus, brain simulations elaborated from static and averaged atlases might be likely useless in simulating brain function. Realistic solutions require that the dynamic entity of the simulated brain “grows” and interacts with the same outside world as the real brain, i.e., that both share the same interactive constraints at any point in time to produce the same behavior or implement the same cognitive process.
Point 2: How do system and computational neurosciences integrate theory so far? In a provocative review (103), Carandini assumes the existence of an intermediate level of circuit integration, where canonical operations can be defined as invariant computations repeated and combined in different ways across the brain. To identify them, it becomes necessary to record from a myriad of neurons in multiple brain regions rather than from single neurons. “Understanding computation…provides a language for theories of behavior.” This concept is very close to the algorithmic level of Marr, because it no longer depends on the understanding of the biophysics of the substrate, which may vary from region to region and species to species. However, most consensual canonical principles are not derived from the search of big data but from philosophical or psychological principles arising from past centuries (115). For instance, the current theories of associative synaptic plasticity did not originate with spike-timing–dependent plasticity (STDP) but can be seen as the revival of causality-based rules inherited from psychologists [(116–118), to cite only a few (119)]. Other rules address a more macroscopic level, irrespective of the biological substrate implementation of the underlying mechanisms, such as the psychic laws of the Gestalt school in 1930s (117, 121) or the binding-by-synchrony hypothesis (120). It is only recently that the introduction of top-down constraints satisfying Bayesian optimization (19, 20) seems to provide innovative insights into mesoscopic processing in the brain and the way it adapts to multiple task-driven constraints.
Point 3: Exploiting biological data obtained at different spatial and temporal scales should benefit from earlier concepts developed in statistical physics. Anderson (122) points out that the field of supraconductivity shows the reductionist fallacy (see section 3: Marr-Poggio conundrum). The ability to reduce everything to simple laws does not imply the ability to start from those laws and reconstruct the whole (the brain in biology, the universe in physics). The constructionist hypothesis breaks down when confronted with scale changes and complexity (123). Anderson summarizes the principle of “symmetry breaking” across scales, as follows: (i) The internal structure of a piece of matter or a living brain need not be symmetrical even if the total state of it is (an argument that mean field theories do not always follow); (ii) the macroscopic state of a large system has less symmetry than that obeyed by the microscopic laws which govern it. “In the so-called N→infinity limit…matter will undergo mathematically sharp, singular ‘phase transitions’ to states in which the microscopic symmetries…are in a sense violated.…Functional structure in a teleogical sense, as opposed to mere crystalline shape, must also be considered a stage, possibly intermediate between crystallinity and information strings, in the hierarchy of broken symmetries.” A rare echo of this principle can be found in a pioneer multiscale model of emergence of local and global features in the early visual system (75, 124, 125).
Progress should be expected by building novel descriptive frameworks which extract—from zillions of measurements—mesoscopic variables, analogous to the concept of quasiparticles in statistical physics. Solid-state physicists successfully developed “middle way” theories (126) that overcome the limitation that equations for particle interactions become impossible to solve or simulate for more than 10 particles. The introduction of a formalism based on virtual quasiparticles may simplify the analytical treatment of long-distance interactions between numerous elementary bound particles, by an equivalent free quasiparticle with shorter interaction. The search for such macroscopic variables could offer an analytic way of treating neural network dynamics and enrich the present mean-field equation formalism. This would allow the building of new kinds of “stereological” models of gray matter, combining the local-range connectivity of columnar ensembles, the extrasynaptic volume diffusion of second messengers and modulators, and the oscillatory coupling due to physical distance in the three-dimensional (3D) brain [a factor unaccounted for by classical ring (1D) or layered (2D) networks]. Quasiparticles have dual corpuscular and wave counterparts, which may apply to information diffusion and propagation across cortical networks, for which evidence can be monitored by fast voltage-sensitive dye imaging. Use of such models may reconcile the physics of interacting particles and waves with the functional physiology of long-distance interconnected cortical columns.
The search for a unified theory, as in particle physics, remains at a rudimentary stage for the brain sciences. When changing scales, symmetry breaks introduce major nonlinearities that we cannot account for at present. Thus, the validity of theories and the choice of the relevant explanatory variables remain restricted to certain levels of integration, resulting in simulation attempts that are essentially local and species- and task-dependent. The hope is that understanding mesoscale organization and full network dynamics might reveal a simpler formalism than the microscale level, similar to general laws in statistical thermodynamics (127). The limitation for reverse engineering is that mean-field-like approaches, because of their underlying simplifications, will lose important generative mechanisms of low-level nonlinearities. A more empirical and modest alternative could be to multiply the diversity of proposed multiscale models, selecting those that most efficiently reduce complexity: “A good theoretical model of a complex system should be like a good caricature: It should emphasize those features which are most important and should downplay the inessential details.… Since one does not really know which are the inessential details until one has understood the phenomena under study…one should investigate a wide range of models and not stake one’s life (or one’s theoretical insight) on one particular model only” (128). Hence, again, the definition of multiscale data integration and the convergence to a theoretical understanding must be progressive and recursive.8. The risks, for basic research, of dominant strategies based on “economics of promises”
Let us leave theory and move to the economics and policy of science. International think-tank meetings for defining a worldwide unified strategy (129, 130) attract public attention and feed the buzz of wide-audience science chronicles. Large-scale brain initiatives are often presented to the public as unselfish but costly science, generating state-of-the-art infrastructures and large data resources open to the community. They are advertised as opening the door for brain-derived information technology (IT) and, in the minds of some high profile IT leaders, paving the way to transhumanism (131, 132).
Part of the original motivation for big data comes from its success in studying simple organisms: for instance, the complete lineage and full reconstruction using electron microscopy of C. elegans, initiated in the 1980s, were shared by the entire field, leading to faster progress. However, the justification for the full human brain simulation is more questionable: The metaphor of “mind observatory,” used rhetorically to link it with physics exploratory platforms such as CERN, is misleading. Megascience infrastructures in physics take immediate advantage of shared “unique” instruments, which have been cooperatively designed to collect new experimental data and test explicit hypotheses through an overarching theory. In the brain sciences, however, building massive database architecture without theoretical guidance may turn into a waste of time and money (133, 134).
The “observatory” function itself, i.e., yielding new data that were formerly out of reach because of technical limitations, is not even central to some of the large-scale brain initiatives. For instance, the flagship project (HBP) transformed its original drive (for a better understanding of brain) into a “viewing neuroscope” IT platform built largely on preexisting data. Progress is expected mostly from an alliance of deep learning, neuroinformatics, and neuromorphic computation, and promised to be quantitative enough to sustain virtual medicine applications (135).
This strategic drift illustrates the impact of “megascience,” considered by sociologists of emergent technologies as a new form of societo-scientific culture (131, 132, 136–139). “Economics of promise” are built around a scientific or industrial process (or even a theoretical law) whose justification is primarily based not on scientific or technological arguments but on the promises themselves (as if these were guaranteed to be fulfilled). This trend, which has deep roots linked to what modern society expects from biology in the large sense, has been repeatedly observed in different scientific subfields such as large-scale brain simulation, nanotechnology, stem cells, and synthesis biology (138). It even applies to the myth of Moore’s law that perpetuates itself because of the marketing of chip designers in neuromorphic computing (132, 140).
Plausible reasons have been identified to justify such drastic changes in scientific conduct: rarefaction of funding for basic research in brain science, the necessary requirement of a major translational impact at the societal level, “hype” purposely designed to reach the largest public audience as well as political decision-makers, overselling promises in the public health domain and possible blue-sky industrial outcomes. The attractiveness to politicians, administrators, and funders (whether public or private) of massive and visible one-track programs is obvious (141), but one may consider that high-level “deciders” are not always entirely aware of —or possibly interested in— the downsides of these mammoth programs, or of the obvious weaknesses of their scientific underpinnings. Promises are no longer an extrapolation of the “possible future” (Fig. 2), but become the scientific justifications of purely economic and political “bubble” strategies engineered to capture funding on the basis of competitive supranational calls (139, 142).
“The present trend prefigures a radical societal change in scientific conduct…”Fig. 2 Building brain sciences through “economics of promises”?
Promises based on data-driven exploration and modeling of the human brain share similarities and even inspiration with the imagery of science fiction. They become the scientific justification for the capture of large-scale funding.CREDIT: ZAP ART/GETTY IMAGES
A side effect is that governmental institutions in Europe and the United States suggest that enough data may be already available on the laboratory shelves, constituting a pile of “siloed” dormant sources that need to be curated (143, 144). Will this become a cheap pretense used to justify budget reduction in experimental basic neuroscience? It seems indeed easier in terms of budget control to turn scientists into high-tech engineers rather than to fund basic research on a wider spectrum with reduced short-term impact.
There exists a real danger that a few large-scale international projects building the foundations of virtual or in silico neuroscience will massively engage the funds available in basic neurosciences to the detriment of small and medium-size basic research initiatives focusing on integrative, cognitive, or computational neuroscience. One gets the impression that the future of acquisition and exploitation of brain-related data will be shared between a few large-scale continental initiatives or strong industrial-like ventures. The possibility of conflicts of interest (which grows with the size of the consortia), of attempts to self-appropriate knowledge and eventually make a profitable business of it (145, 146), all remind us that it is urgent to define worldwide accepted standards of transparent macro-management and access to data and technologies.Conclusion
In this Review, I have tried to point out that, although big-data and technological advances undeniably have immense value for future developments, the expedient industrialization of neuroscience and the potential long-term importance of the personal, political, and commercial incentives driving it are causes for concern. Systematic and streamlined approaches are not appropriate for all facets of brain research, and the interpretation of massive data sets collected without appropriate forethought may turn out to be impossible. Given the exponentially increasing rate at which big data are being collected, exabyte information will be accumulated before the end of the next decade. Out of this magma, it may be difficult to tease out of the hypothetical key principles that might help resolve the main questions that should have been at the root of their design and made explicit all along.
Megascience dominance, if improperly managed, may lead to the drying up of traditional funding channels and the disappearance of smaller-scale and rationally designed research programs, which are still the major source of breaking discoveries. To master megascience development and reduce negative side effects, current strategies could be greatly improved by the following:
1) rationalizing the codesign of the choice of experimental models (choice of species, precise targeting of behavioral specificity) and the justification of appropriate techniques (sensitivity range of the instrumentation, spatial and temporal scale ranges to be explored);
2) clarifying the hidden scientific assumptions associated with each instrumentation type and interrelating explanatory variables (i.e., conductance, spike rate, calcium fluorescence, metabolic or hemodynamic signals) despite their biophysical diversity;
3) clarifying the hidden impact of preprocessing steps and statistical methods to reduce across-study heterogeneity;
4) developing more efficient recursive loops between experiments and theory-driven top-down predictions, to confront a larger diversity of brain models and compare their predictive power;
5) building innovative theoretical frameworks not only inspired by computational neuroscience, mathematics, and psychology, but also enriched by complementary fields used to deal with complex systems of high dimensionality (statistical physics, thermodynamics, astrophysics);
6) vetting the most relevant experimental paradigms, to define in an unbiased way the parametric features and the reproducibility of the stimulation context necessary to the constitution of large–data set repositories;
7) allowing open access—to scientists and modelers—to the entire data reservoir and its data sharing, devoid of selective control by the ownership claims of grant funders.
These changes in scientific planning will undoubtedly require the generalized practice of interdisciplinarity between physics and biology, focusing on the major bottlenecks (129, 130). Only in this way, can we hope to improve our critical skills and collectively optimize our capacity to better anticipate the challenges we face in exploring uncharted levels of complexity.Conceptual illustration: The Mind-Body Problem.CREDIT: ARTWORK: EBERHARDT E. FETZ, COURTESY WASHINGTON UNIVERSITY References and Notes
D. Le Bihan, Looking Inside the Brain: The Power of Neuroimaging (Princeton Univ. Press, NJ, 2014).
F. Dyson, Imagined worlds. The Jerusalem-Harvard Lectures (Harvard Univ. Press, Cambridge, 1997).
C. Lange, in Nobel Lectures, Peace, 1901-1925, F. Haberman, Ed. (Elsevier, Amsterdam, 1972).
D. Marr, Vision (MIT Press, Cambridge, 1982).
T. Poggio, Visual Algorithms (MIT, Cambridge, 1982).
J. A. Bednar, C. K. I. Williams, in From Neuron to Cognition via Computational Neuroscience, M. A. Arbib, J. J. Bonaiuto, Eds. (MIT Press, Cambridge, 2016), pp. 409–432.
P. Dayan, L. Abbott, Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems (MIT Press, Cambridge, 2002).
B. Olshausen, in 20 Years of Computational Neuroscience, J. M. Bower, D. Beeman, Eds. (Springer, New York, 2013).
J. M. Bower, D. Beeman, The Book of GENESIS: Exploring Realistic Neural Models with the GEneral NEural SImulation System (Telos, New York, 1998).
C. von der Malsburg, in Brain Theory, G. Palm, A. Aertsen, Eds. (Springer, Berlin, 1986), pp. 161–176.
Y. Fregnac, Big science needs big concepts, in “Voices”: BRAIN Initiative and Human Brain Project: Hopes and reservations. Cell 155, 265–266 (2013).10.1016/j.cell.2013.09.037
W. James, Psychology: Briefer Course (Harvard Univ. Press, Cambridge, 1890).
Y. Delage, Le Rève: Etude Psychologique, Philosophique et Litteraire [The Dream: A Psychological, Philosophical and Literary Study (in French)] (Presses Universitaires de France, Paris, 1919).
D. Hebb, The Organization of Behavior (Wiley, New York, 1949).
V. Y. Frenkel, Yakov Ilich Frenkel: His Work, Life, and Letters (Birkhäuser Verlag, Basel/Boston, 1996).
L. Ferry, La révolution transhumaniste [The Revolution of “Transhumanism”]. (Plon, Paris, 2016).
J.-G. Ganascia, Le mythe de la singularité [The Myth of Singularity (in French)]. Science Ouverte (Seuil, Paris, 2017).
U. Felt, B. Wyne, “Taking European knowledge society seriously,” Report of the Expert Group on Science and Governance to the Science, Economy and Society Directorate, Directorate-General for Research (European Commission, Brussels, 2007).
Sciences et Technologies émergentes: pourquoi tant de promesses? M. Audetat, Ed., Emerging Sciences and Technologies (Hermann, 2015).
F. Panese, in Sciences et Technologies émergentes: pourquoi tant de promesses, M. Audétat, Ed. (Hermann, Paris, 2015), pp. 165–193.
S. Loeve, in Sciences et Technologies émergentes: pourquoi tant de promesses? M. Audetat, Ed. (Hermann, Paris, 2015), pp. 91–113.
3COM [8 Certification Exam(s) ]
AccessData [1 Certification Exam(s) ]
ACFE [1 Certification Exam(s) ]
ACI [3 Certification Exam(s) ]
Acme-Packet [1 Certification Exam(s) ]
ACSM [4 Certification Exam(s) ]
ACT [1 Certification Exam(s) ]
Admission-Tests [13 Certification Exam(s) ]
ADOBE [93 Certification Exam(s) ]
AFP [1 Certification Exam(s) ]
AICPA [2 Certification Exam(s) ]
AIIM [1 Certification Exam(s) ]
Alcatel-Lucent [13 Certification Exam(s) ]
Alfresco [1 Certification Exam(s) ]
Altiris [3 Certification Exam(s) ]
Amazon [2 Certification Exam(s) ]
American-College [2 Certification Exam(s) ]
Android [4 Certification Exam(s) ]
APA [1 Certification Exam(s) ]
APC [2 Certification Exam(s) ]
APICS [2 Certification Exam(s) ]
Apple [69 Certification Exam(s) ]
AppSense [1 Certification Exam(s) ]
APTUSC [1 Certification Exam(s) ]
Arizona-Education [1 Certification Exam(s) ]
ARM [1 Certification Exam(s) ]
Aruba [6 Certification Exam(s) ]
ASIS [2 Certification Exam(s) ]
ASQ [3 Certification Exam(s) ]
ASTQB [8 Certification Exam(s) ]
Autodesk [2 Certification Exam(s) ]
Avaya [96 Certification Exam(s) ]
AXELOS [1 Certification Exam(s) ]
Axis [1 Certification Exam(s) ]
Banking [1 Certification Exam(s) ]
BEA [5 Certification Exam(s) ]
BICSI [2 Certification Exam(s) ]
BlackBerry [17 Certification Exam(s) ]
BlueCoat [2 Certification Exam(s) ]
Brocade [4 Certification Exam(s) ]
Business-Objects [11 Certification Exam(s) ]
Business-Tests [4 Certification Exam(s) ]
CA-Technologies [21 Certification Exam(s) ]
Certification-Board [10 Certification Exam(s) ]
Certiport [3 Certification Exam(s) ]
CheckPoint [41 Certification Exam(s) ]
CIDQ [1 Certification Exam(s) ]
CIPS [4 Certification Exam(s) ]
Cisco [318 Certification Exam(s) ]
Citrix [48 Certification Exam(s) ]
CIW [18 Certification Exam(s) ]
Cloudera [10 Certification Exam(s) ]
Cognos [19 Certification Exam(s) ]
College-Board [2 Certification Exam(s) ]
CompTIA [76 Certification Exam(s) ]
ComputerAssociates [6 Certification Exam(s) ]
Consultant [2 Certification Exam(s) ]
Counselor [4 Certification Exam(s) ]
CPP-Institue [2 Certification Exam(s) ]
CPP-Institute [1 Certification Exam(s) ]
CSP [1 Certification Exam(s) ]
CWNA [1 Certification Exam(s) ]
CWNP [13 Certification Exam(s) ]
Dassault [2 Certification Exam(s) ]
DELL [9 Certification Exam(s) ]
DMI [1 Certification Exam(s) ]
DRI [1 Certification Exam(s) ]
ECCouncil [21 Certification Exam(s) ]
ECDL [1 Certification Exam(s) ]
EMC [129 Certification Exam(s) ]
Enterasys [13 Certification Exam(s) ]
Ericsson [5 Certification Exam(s) ]
ESPA [1 Certification Exam(s) ]
Esri [2 Certification Exam(s) ]
ExamExpress [15 Certification Exam(s) ]
Exin [40 Certification Exam(s) ]
ExtremeNetworks [3 Certification Exam(s) ]
F5-Networks [20 Certification Exam(s) ]
FCTC [2 Certification Exam(s) ]
Filemaker [9 Certification Exam(s) ]
Financial [36 Certification Exam(s) ]
Food [4 Certification Exam(s) ]
Fortinet [13 Certification Exam(s) ]
Foundry [6 Certification Exam(s) ]
FSMTB [1 Certification Exam(s) ]
Fujitsu [2 Certification Exam(s) ]
GAQM [9 Certification Exam(s) ]
Genesys [4 Certification Exam(s) ]
GIAC [15 Certification Exam(s) ]
Google [4 Certification Exam(s) ]
GuidanceSoftware [2 Certification Exam(s) ]
H3C [1 Certification Exam(s) ]
HDI [9 Certification Exam(s) ]
Healthcare [3 Certification Exam(s) ]
HIPAA [2 Certification Exam(s) ]
Hitachi [30 Certification Exam(s) ]
Hortonworks [4 Certification Exam(s) ]
Hospitality [2 Certification Exam(s) ]
HP [750 Certification Exam(s) ]
HR [4 Certification Exam(s) ]
HRCI [1 Certification Exam(s) ]
Huawei [21 Certification Exam(s) ]
Hyperion [10 Certification Exam(s) ]
IAAP [1 Certification Exam(s) ]
IAHCSMM [1 Certification Exam(s) ]
IBM [1532 Certification Exam(s) ]
IBQH [1 Certification Exam(s) ]
ICAI [1 Certification Exam(s) ]
ICDL [6 Certification Exam(s) ]
IEEE [1 Certification Exam(s) ]
IELTS [1 Certification Exam(s) ]
IFPUG [1 Certification Exam(s) ]
IIA [3 Certification Exam(s) ]
IIBA [2 Certification Exam(s) ]
IISFA [1 Certification Exam(s) ]
Intel [2 Certification Exam(s) ]
IQN [1 Certification Exam(s) ]
IRS [1 Certification Exam(s) ]
ISA [1 Certification Exam(s) ]
ISACA [4 Certification Exam(s) ]
ISC2 [6 Certification Exam(s) ]
ISEB [24 Certification Exam(s) ]
Isilon [4 Certification Exam(s) ]
ISM [6 Certification Exam(s) ]
iSQI [7 Certification Exam(s) ]
ITEC [1 Certification Exam(s) ]
Juniper [64 Certification Exam(s) ]
LEED [1 Certification Exam(s) ]
Legato [5 Certification Exam(s) ]
Liferay [1 Certification Exam(s) ]
Logical-Operations [1 Certification Exam(s) ]
Lotus [66 Certification Exam(s) ]
LPI [24 Certification Exam(s) ]
LSI [3 Certification Exam(s) ]
Magento [3 Certification Exam(s) ]
Maintenance [2 Certification Exam(s) ]
McAfee [8 Certification Exam(s) ]
McData [3 Certification Exam(s) ]
Medical [69 Certification Exam(s) ]
Microsoft [374 Certification Exam(s) ]
Mile2 [3 Certification Exam(s) ]
Military [1 Certification Exam(s) ]
Misc [1 Certification Exam(s) ]
Motorola [7 Certification Exam(s) ]
mySQL [4 Certification Exam(s) ]
NBSTSA [1 Certification Exam(s) ]
NCEES [2 Certification Exam(s) ]
NCIDQ [1 Certification Exam(s) ]
NCLEX [2 Certification Exam(s) ]
Network-General [12 Certification Exam(s) ]
NetworkAppliance [39 Certification Exam(s) ]
NI [1 Certification Exam(s) ]
NIELIT [1 Certification Exam(s) ]
Nokia [6 Certification Exam(s) ]
Nortel [130 Certification Exam(s) ]
Novell [37 Certification Exam(s) ]
OMG [10 Certification Exam(s) ]
Oracle [279 Certification Exam(s) ]
P&C [2 Certification Exam(s) ]
Palo-Alto [4 Certification Exam(s) ]
PARCC [1 Certification Exam(s) ]
PayPal [1 Certification Exam(s) ]
Pegasystems [12 Certification Exam(s) ]
PEOPLECERT [4 Certification Exam(s) ]
PMI [15 Certification Exam(s) ]
Polycom [2 Certification Exam(s) ]
PostgreSQL-CE [1 Certification Exam(s) ]
Prince2 [6 Certification Exam(s) ]
PRMIA [1 Certification Exam(s) ]
PsychCorp [1 Certification Exam(s) ]
PTCB [2 Certification Exam(s) ]
QAI [1 Certification Exam(s) ]
QlikView [1 Certification Exam(s) ]
Quality-Assurance [7 Certification Exam(s) ]
RACC [1 Certification Exam(s) ]
Real-Estate [1 Certification Exam(s) ]
RedHat [8 Certification Exam(s) ]
RES [5 Certification Exam(s) ]
Riverbed [8 Certification Exam(s) ]
RSA [15 Certification Exam(s) ]
Sair [8 Certification Exam(s) ]
Salesforce [5 Certification Exam(s) ]
SANS [1 Certification Exam(s) ]
SAP [98 Certification Exam(s) ]
SASInstitute [15 Certification Exam(s) ]
SAT [1 Certification Exam(s) ]
SCO [10 Certification Exam(s) ]
SCP [6 Certification Exam(s) ]
SDI [3 Certification Exam(s) ]
See-Beyond [1 Certification Exam(s) ]
Siemens [1 Certification Exam(s) ]
Snia [7 Certification Exam(s) ]
SOA [15 Certification Exam(s) ]
Social-Work-Board [4 Certification Exam(s) ]
SpringSource [1 Certification Exam(s) ]
SUN [63 Certification Exam(s) ]
SUSE [1 Certification Exam(s) ]
Sybase [17 Certification Exam(s) ]
Symantec [134 Certification Exam(s) ]
Teacher-Certification [4 Certification Exam(s) ]
The-Open-Group [8 Certification Exam(s) ]
TIA [3 Certification Exam(s) ]
Tibco [18 Certification Exam(s) ]
Trainers [3 Certification Exam(s) ]
Trend [1 Certification Exam(s) ]
TruSecure [1 Certification Exam(s) ]
USMLE [1 Certification Exam(s) ]
VCE [6 Certification Exam(s) ]
Veeam [2 Certification Exam(s) ]
Veritas [33 Certification Exam(s) ]
Vmware [58 Certification Exam(s) ]
Wonderlic [2 Certification Exam(s) ]
Worldatwork [2 Certification Exam(s) ]
XML-Master [3 Certification Exam(s) ]
Zend [6 Certification Exam(s) ]