- 24 Jun, 2020 1 commit
-
-
Jeff Zhang authored
### What is this PR for? Simple PR to upgrade to official released spark 3.0.0. ### What type of PR is it? [ Improvement ] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4893 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3809 from zjffdu/ZEPPELIN-4893 and squashes the following commits: e34cd7984 [Jeff Zhang] save 0918aebfb [Jeff Zhang] [ZEPPELIN-4893]. Upgrade to spark 3.0.0
-
- 23 Jun, 2020 4 commits
-
-
Philipp Dallig authored
### What is this PR for? This PR fixes the problem, that hadoop libraries are not in every submodule. ### What type of PR is it? - Improvement ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4902 ### How should this be tested? * **Travis-CI**: https://travis-ci.org/github/Reamer/zeppelin/builds/700094748 ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Philipp Dallig <philipp.dallig@gmail.com> Closes #3817 from Reamer/pom_cleanup and squashes the following commits: fa221660a [Philipp Dallig] We need kerberos-libraries with hadoop3 54db1215d [Philipp Dallig] Plugins should not need extra hadoop jars 933069ca3 [Philipp Dallig] Remove global dependency for all submodules, if you run with hadoop3 profile d2e4569cc [Philipp Dallig] test dependencies should be in test scope
-
Alex Ott authored
### What is this PR for? New DataStax Java Driver allows to override configuration options by different ways - via config file, system properties, etc. This PR implements following: * Allow to override any option of the Java driver by putting additional options into interpreter's configuration. For example, this allows to use DataStax Astra (Cassandra as a Service) without need to make any changes in the code of the interpreter. It also allows to configure any additional parameter not exposed directly by interpreter * Upgrades driver to latest released version * Configures embedded Cassandra for faster startup when executing unit tests ### What type of PR is it? Improvement ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4818 ### How should this be tested? * https://travis-ci.org/github/alexott/zeppelin/builds/700969578 * tested manually Author: Alex Ott <alexott@gmail.com> Closes #3815 from alexott/ZEPPELIN-4818 and squashes the following commits: f34738f75 [Alex Ott] [ZEPPELIN-4818] Upgrade to driver 4.7.2 5ef5c8823 [Alex Ott] [ZEPPELIN-4818] Allow to provide any Java driver setting a458df7ba [Alex Ott] [ZEPPELIN-4818] Explicitly set timeout for DDL statements 42dc4e65c [Alex Ott] [ZEPPELIN-4818] Upgrade driver to 4.7.1 db3950f1b [Alex Ott] [ZEPPELIN-4818] Speedup unit test
-
Nathaniel Troutman authored
### What is this PR for? When S3NotebookRepo saves a notebook into S3 it does so without granting the bucket owner full control of the object, this means that owning AWS account cannot read the notebook, only the creating AWS account. This causes issues when a notebook bucket is shared across AWS accounts. This PR introduces the configuration value "zeppelin.notebook.s3.cannedAcl" which will override the default object permissions when saving a notebook to S3. This allows granting the bucket owner full control when the writer and the bucket owner are different AWS accounts. ### What type of PR is it? Improvement/Feature ### Todos * [ ] - N/A ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-4896 ### How should this be tested? #### Unit-Tests Unfortunately Unit-Tests are not possible as gaul.S3Proxy does not support Object ACL throwing a NotImplemented exception if you attempt to use PutRequest.withCannedAcl for any value other than public or private (see https://github.com/gaul/s3proxy#limitations). #### Manual Testing 1. Have two AWS accounts, "account-a" and "account-b" 2. Create a bucket "notebook-bucket" in "account-a" and grant "account-b" permissions to write to it 3. Setup zeppelin-site.xml for S3NotebookRepor WITHOUT Canned ACL Feature and AWS Credentials for "account-b" 4. Launch zeppelin, create a new note titled 'WithoutACL', and shutdown zeppelin 5. Verify that the permissions on the S3 Object do NOT grant the bucket owner "account-a" permissions ``` aws s3api get-object-acl --bucket notebook-bucket --key test-user/notebook/WithoutACL_2FD4NFYTU.zpln { "Owner": { "DisplayName": "account-b", "ID": "1e9...e4" }, "Grants": [ { "Grantee": { "DisplayName": "account-b", "ID": "1e9...e4", "Type": "CanonicalUser" }, "Permission": "FULL_CONTROL" } ] } ``` 6. Edit zeppelin-site.xml enabling "zeppelin.notebook.s3.cannedAcl" as "BucketOwnerFullControl" 7. Launch zeppelin, create a new note titled "WithACL", and shutdown zeppelin 8. Verify that the permissions on the S3 object DO GRANT the bucket owner, "account-a", full control ``` aws s3api get-object-acl --bucket notebook-bucket --key test-user/notebook/WithACL_2FCXTUS3M.zpln { "Owner": { "DisplayName": "account-b", "ID": "1e9...e4" }, "Grants": [ { "Grantee": { "DisplayName": "account-b", "ID": "1e9...e4", "Type": "CanonicalUser" }, "Permission": "FULL_CONTROL" }, { "Grantee": { "DisplayName": "account-a", "ID": "f60...ee", "Type": "CanonicalUser" }, "Permission": "FULL_CONTROL" } ] } ``` ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this need documentation? Yes, added relevant documentation along side existing S3 Notebook Repo documentation. Author: Nathaniel Troutman <troutma@amazon.com> Closes #3811 from ntroutman/canned-acl and squashes the following commits: 6ebe608a1 [Nathaniel Troutman] [ZEPPELIN-4896] Add S3 Canned ACL Option for S3NotebookRepo
-
Jeff Zhang authored
### What is this PR for? Trivial PR to create these injected variables when starting zeppelin server. Now these variables are created when user open zeppelin in browser if we don't get it explicitly here. Lazy loading will cause paragraph recovery and cron job initialization is delayed. ### What type of PR is it? [Improvement ] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4899 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3816 from zjffdu/ZEPPELIN-4899 and squashes the following commits: 32e47278e [Jeff Zhang] [ZEPPELIN-4899]. Injected variables in ZeppelinServer are lazy loaded
-
- 22 Jun, 2020 2 commits
-
-
Jeff Zhang authored
### What is this PR for? This PR update the flink interpreter doc for the recently improvement in flink interpreter and add some screenshot. ### What type of PR is it? [Documentation] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4839 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3779 from zjffdu/ZEPPELIN-4839 and squashes the following commits: ddc774078 [Jeff Zhang] [ZEPPELIN-4839]. Update flink interpreter doc
-
Jeff Zhang authored
### What is this PR for? Add document explaining what is interpreter recovery and how to use it. ### What type of PR is it? [ Documentation ] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4875 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3803 from zjffdu/ZEPPELIN-4875 and squashes the following commits: 9c2e71ef0 [Jeff Zhang] [ZEPPELIN-4875]. Add document for interpreter recovery
-
- 20 Jun, 2020 1 commit
-
-
Jeff Zhang authored
-
- 19 Jun, 2020 2 commits
-
-
Alex Ott authored
### What is this PR for? Refactoring of the Cassandra interpreter to a new driver changes the results representation to be strict CQL format, that is not always desirable. This PR improves the situation by allow to control the output format (CQL vs human-readable), and also control such things as locale, timezone, format of timestamp/date/time columns, and precision for float & double types. All settings could be set either on interpreter, or cell level. ### What type of PR is it? Improvement ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4796 ### How should this be tested? * https://travis-ci.org/github/alexott/zeppelin/builds/698160753 * Tested manually (see screenshot) ### Screenshots (if appropriate)  Author: Alex Ott <alexott@gmail.com> Closes #3790 from alexott/ZEPPELIN-4796 and squashes the following commits: 6e4fcc9eb [Alex Ott] [ZEPPELIN-4796] Added cell-level formatting, and documentation e008982a6 [Alex Ott] [ZEPPELIN-4796] initial implementation of new formatting
-
Jeff Zhang authored
-
- 18 Jun, 2020 6 commits
-
-
Philipp Dallig authored
### What is this PR for? This PR will delete the content of the plugin folders. Unfortunately the main folder is still present after a `mvn clean` ### What type of PR is it? - Bug Fix ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4892 ### How should this be tested? * **Travis-CI:** https://travis-ci.org/github/Reamer/zeppelin/builds/699667326 ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Philipp Dallig <philipp.dallig@gmail.com> Closes #3808 from Reamer/clean_plugin_dir and squashes the following commits: a0dab671c [Philipp Dallig] Delete plugin dir in clean phase
-
Philipp Dallig authored
### What is this PR for? Fixes a Maven warning because maven-resources-plugin and net.jodah:concurrentunit is defined twice in flink/interpreter/pom.xml ### What type of PR is it? Bug Fix ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4891 ### How should this be tested? * **Travis-CI**: https://travis-ci.org/github/Reamer/zeppelin/builds/699634998 ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Philipp Dallig <philipp.dallig@gmail.com> Closes #3807 from Reamer/mvn_warning_flink_interpreter and squashes the following commits: f5e945060 [Philipp Dallig] Fixes a Maven warning because maven-resources-plugin and net.jodah:concurrentunit is defined twice in this pom.xml
-
Jeff Zhang authored
### What is this PR for? This PR to support single row result displaying in frontend. For now we always use table format for sql output, but sometimes user want to just display a single value or single row in frontend (e.g. displaying KPI value). This PR use `SingleRowInterpreterResult` to represent such sql output, and refactor existing code to leverage this class to display single row in frontend. Besides that this PR also support the single row result in spark sql. ### What type of PR is it? [ Improvement | Feature | Documentation ] ### Todos * [ ] - Task ### What is the Jira issue? * https://github.com/zjffdu/zeppelin/compare/ZEPPELIN-4850 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3791 from zjffdu/ZEPPELIN-4850 and squashes the following commits: 10b992e70 [Jeff Zhang] [ZEPPELIN-4850]. Include charting option for single value
-
Alex Ott authored
### What is this PR for? Spark interpreter did use incorrect property name for Spark Master - `master`, although a lot of code was dependent on `spark.master` - it's better to use only one name everywhere ### What type of PR is it? Bug Fix ### What is the Jira issue? * ZEPPELIN-4821 ### How should this be tested? * https://travis-ci.org/github/alexott/zeppelin/builds/699582429 Author: Alex Ott <alexott@apache.org> Closes #3773 from alexott/ZEPPELIN-4821 and squashes the following commits: 82b8321f1 [Alex Ott] [ZEPPELIN-4821] Spark interpreter uses incorrect property name
-
dijie authored
### What is this PR for? Fix error logs in catch exception ### What type of PR is it? [Bug Fix ] ### Todos * [ ] - Task ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-4889 ### How should this be tested? ### Screenshots (if appropriate) ### Questions: No questions. Author: dijie <dijie@mogu.com> Closes #3804 from lonelyGhostisdog/ZEPPELIN-4889 and squashes the following commits: 636b69809 [dijie] [ZEPPELIN-4889] [Flink] Error logs in catch exception
-
Jeff Zhang authored
-
- 17 Jun, 2020 5 commits
-
-
Alex Ott authored
### What is this PR for? Right now, when operation returns no results, for example, when executing most operations other than `SELECT` (DDL/INSERT/UPDATE/DELETE/...), interpreter shows the extended information - what hosts were used to execute command, etc. This information occupies a significant amount of the screen space and isn't critical for execution. Given PR provides the same information but folds it by default, saving the screen space. ### What type of PR is it? Improvement ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4756 ### How should this be tested? * https://travis-ci.org/github/alexott/zeppelin/builds/698161348 ### Screenshots (if appropriate) Old view:  New view:   Author: Alex Ott <alexott@gmail.com> Closes #3801 from alexott/ZEPPELIN-4756 and squashes the following commits: 3b0618c5a [Alex Ott] [ZEPPELIN-4756] Cassandra: Show "No Results" message folded
-
Jeff Zhang authored
-
Jeff Zhang authored
### What is this PR for? It is a trivial PR which add property `zeppelin.flink.module.enableHive` to control whether load hive udf first when there's same udf name in both hive and flink. ### What type of PR is it? [Improvement] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4885 ### How should this be tested? * CI pass, and manually tested. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3802 from zjffdu/ZEPPELIN-4885 and squashes the following commits: 45754580f [Jeff Zhang] [ZEPPELIN-4885]. Add property to specify Module order of flink interpreter
-
Jeff Zhang authored
### What is this PR for? Trivial PR which add property `flink.webui.yarn.yarnAddress` that allow user to replace the yarn address. This is for some cases that cloud vender would map the internal resource manager address to other url. e.g. some cloud vender will map `http://resource-manager:8088` to `https://xxx-yarn.yy.cn/gateway/kkk/yarn` ### What type of PR is it? [Improvement] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4851 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Jeff Zhang <zjffdu@apache.org> Closes #3797 from zjffdu/ZEPPELIN-4851 and squashes the following commits: c1be76217 [Jeff Zhang] [ZEPPELIN-4851]. Add property to flink interpreter to allow replace the yarn address
-
Jeff Zhang authored
### What is this PR for? This PR fix the bug in Splitter that text between 2 sql comment is mistaken as comment. Unit test is added in this PR as well. Besides that, this PR also remove the comment in the middle of sql. e.g. ``` select a -- comment 1 from table_1 ``` will be converted to ``` select a from table_1 ``` ### What type of PR is it? [Bug Fix] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4877 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3796 from zjffdu/ZEPPELIN-4877 and squashes the following commits: 0b0b72802 [Jeff Zhang] address comment 6e9287e16 [Jeff Zhang] address comment 63afd6ad3 [Jeff Zhang] [ZEPPELIN-4877]. text between 2 sql comment is mistaken as comment
-
- 16 Jun, 2020 4 commits
-
-
Jeff Zhang authored
### What is this PR for? Flink interpreter doesn't work with flink 1.10.1 in yarn mode due to FLINK-17788, this PR fix it in zeppelin side. Because in scala shell, flink only support yarn session mode, so when it is yarn mode, we can set deployment target to be yarn-session directly. ### What type of PR is it? [Bug Fix ] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4878 ### How should this be tested? * CI pass and manually tested ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3795 from zjffdu/ZEPPELIN-4878 and squashes the following commits: 498325cc8 [Jeff Zhang] use flink1.10.version & flink1.11.version fb1a761b5 [Jeff Zhang] [ZEPPELIN-4878]. Unable to run flink 1.10.1 in yarn mode due to FLINK-17788
-
Jeff Zhang authored
### What is this PR for? Add property `flink.udf.jars.packages` to specify the packages that would be searched for udf, otherwise all the classes will be check, which might be very time consuming especially when the udf jars are large (if your udf depends on other third party libraries) ### What type of PR is it? [Feature ] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4837 ### How should this be tested? * CI pass and manually tested ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3798 from zjffdu/ZEPPELIN-4837 and squashes the following commits: 78816e548 [Jeff Zhang] [ZEPPELIN-4837]. Add property to only check some packages for udf finding
-
Jeff Zhang authored
### What is this PR for? Trivial PR which display rich duration info instead of just x seconds. See screenshot below. ### What type of PR is it? [ Improvement ] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4873 ### How should this be tested? * CI pass ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3794 from zjffdu/ZEPPELIN-4873 and squashes the following commits: 459f181c5 [Jeff Zhang] add java doc 419818e4a [Jeff Zhang] address comment 9691eae82 [Jeff Zhang] [ZEPPELIN-4873]. Display rich duration info for insert into flink job
-
Jeff Zhang authored
### What is this PR for? This PR is to support yarn interpreter launch mode, that means to launch the interpreter process in yarn container. So that we can mitigate the memory pressure of zeppelin server machine. This PR add new launcher module `yarn` for that. I manually tested shell, python, jdbc and flink interpreted. ### What type of PR is it? [Feature] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4844 ### How should this be tested? * CI pass and manually tested ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3786 from zjffdu/ZEPPELIN-4844 and squashes the following commits: 899c4e1c0 [Jeff Zhang] [ZEPPELIN-4844]. Support yarn interpreter launch mode
-
- 14 Jun, 2020 1 commit
-
-
Alex Ott authored
### What is this PR for? Tests didn't cover all of the edge cases, that lead to error when parsing specific message. Also, was found that parsing is also duplicated in the `Paragraph` class - it was replaced with call to `ParagraphTextParser`. The most visible change is that `%interpreter(some text` will know fail as there is no matching `)` character. But it's works the same if we change it to `%interpreter (some text`... ### What type of PR is it? Hot Fix ### What is the Jira issue? * ZEPPELIN-4870 ### How should this be tested? * Added one more test * https://travis-ci.org/github/alexott/zeppelin/builds/697944179 Author: Alex Ott <alexott@gmail.com> Closes #3800 from alexott/ZEPPELIN-4870-hotfix and squashes the following commits: 5942c5d35 [Alex Ott] [ZEPPELIN-4870][hotfix] fix edge case, and remove duplicate parser
-
- 12 Jun, 2020 1 commit
-
-
Alex Ott authored
### What is this PR for? We can provide properties that are local to the paragraph, that could be used to pass an additional information for interpreter that could affect its behavior. Unfortunately existing parsing functionality relies on the fact that key/value pairs need to be separated by `,` character, and doesn't handle values with special characters (`,`, `=`, ...) inside, like this: ``` %cassandra(locale=ruRU, timeFormat="E, d MMM yy", floatPrecision = 5, outputFormat=cql) ``` This PR changes the parsing logic to perform character-by-character parsing, and handling of the quoted keys & values, escaping of the special characters, etc. ### What type of PR is it? Improvement ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4870 ### How should this be tested? * https://travis-ci.org/github/alexott/zeppelin/builds/697522260 * additional unit tests were added Author: Alex Ott <alexott@gmail.com> Closes #3799 from alexott/ZEPPELIN-4870 and squashes the following commits: 5fb6ee84e [Alex Ott] [ZEPPELIN-4870] Improve parsing of the paragraph properties
-
- 10 Jun, 2020 4 commits
-
-
Jeff Zhang authored
### What is this PR for? Minor PR which allow user to specify job name as paragraph local properties, by default it is the sql statement if user don't specify it. ### What type of PR is it? [Improvement] ### Todos * [ ] - Task ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-4865 ### How should this be tested? * CI pass and manually tested. ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3792 from zjffdu/ZEPPELIN-4865 and squashes the following commits: 7c6e2f5c9 [Jeff Zhang] [ZEPPELIN-4865]. Allow specify jobName as paragraph local properties
-
Alex Ott authored
### What is this PR for? Existing Travis-CI config contained references to previous distributions, etc. This PR fixes this. It also fixes a problem with Livy interpreter, where tests were failing because R wasn't enabled for them ### What type of PR is it? Bug Fix / Improvement ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4864 ### How should this be tested? * https://travis-ci.org/github/alexott/zeppelin/builds/696385230 Author: Alex Ott <alexott@apache.org> Closes #3793 from alexott/ZEPPELIN-4864 and squashes the following commits: 3fccb1465 [Alex Ott] [ZEPPELIN-4864] enable R for Livy tests 91a30a303 [Alex Ott] [ZEPPELIN-4864] improve debugability of Livy integration test 82233446e [Alex Ott] [ZEPPELIN-4864] Cleanup Travis-CI config
-
Jeff Zhang authored
### What is this PR for? Although flink 1.11 is not released yet, it is already feature cut, so its api is stable now. This PR is to make flink interpreter of zeppelin to support flink 1.11. Due to api change of flink 1.11, this PR introduce flink-shim which will use different api for different version of flink. ### What type of PR is it? [Feature ] ### Todos * [ ] - Task ### What is the Jira issue? * https://jira.apache.org/jira/browse/ZEPPELIN-4816 ### How should this be tested? * CI pass and tested on flink 1.11 snapshot ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? no * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3782 from zjffdu/ZEPPELIN-4816 and squashes the following commits: 6ce065221 [Jeff Zhang] [ZEPPELIN-4816]. Support Flink 1.11
-
Jeff Zhang authored
### What is this PR for? This PR is to recover the running paragraph when recovery is enabled. 1. Get all the running Interpreter processes from RecoveryStorage. 2. Reconnect to the running interpreter process. 3. Resubmit the paragraph to interpreter process. ### What type of PR is it? [ Feature] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4845 ### How should this be tested? * CI pass and manully tested the recovery scenairo. ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3781 from zjffdu/ZEPPELIN-4845 and squashes the following commits: 2302471de [Jeff Zhang] [ZEPPELIN-4845]. Recover running paragraph when recovery is enabled
-
- 09 Jun, 2020 1 commit
-
-
Jeff Zhang authored
### What is this PR for? This is a improvement and refactoring PR which just use PooledObject for the connection from interpreter process to zeppelin server. Otherwise, once the connection is lost, interpreter process can never connect to zeppelin sever again. In this PR, I create class `PooledRemoteClient` which represent the pooled connection objects and this class will manage the life cycle of the connection between thrift client to thrift server. ### What type of PR is it? [ Improvement | Refactoring] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4859 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3788 from zjffdu/ZEPPELIN-4859 and squashes the following commits: 1c965ce7f [Jeff Zhang] [ZEPPELIN-4859]. Use PooledObject for the connection from interpreter process to zeppelin server
-
- 08 Jun, 2020 3 commits
-
-
Jeff Zhang authored
### What is this PR for? Previous plugin loading strategy is to load them from system classloader first, if fails, then fallback to plugin folder classloader. This would produce misleading logging which make user think that the plugin is failed to load. This PR would load plugin from system classloader only when they are builtin plugins. ### What type of PR is it? [Improvement] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4833 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3784 from zjffdu/ZEPPELIN-4833 and squashes the following commits: 62cd4db59 [Jeff Zhang] [ZEPPELIN-4833] misleading logging when fail to load plugin from classpath directly
-
Jeff Zhang authored
### What is this PR for? This PR add paragraph local property `printREPLOutput` for spark interpreter, so that user can switch for enabling Spark REPL output for each paragraph, e.g. ``` %spark(printREPLOutput=false) System.out.println("hello ") println("hello world") ``` ### What type of PR is it? [ Feature ] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4848 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3787 from zjffdu/ZEPPELIN-4848 and squashes the following commits: b6997d99b [Jeff Zhang] [ZEPPELIN-4848]. Switch for enabling Spark REPL output for each paragraph -
Jeff Zhang authored
### What is this PR for? This is a trivial PR which add `getInterpreterGroupId` to `InterpreterClient`, and use `interpreterGroupId` as the identifier of RemoteInterpreterProcess because `interpreterGroupId` is also unique identifier of InterpreterGroup ### What type of PR is it? [Improvement] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4852 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3785 from zjffdu/ZEPPELIN-4852 and squashes the following commits: 01ec89373 [Jeff Zhang] address comment 4fc92cc60 [Jeff Zhang] address comment a482faa39 [Jeff Zhang] [ZEPPELIN-4852]. Add name to RemoteInterpreterProcess
-
- 07 Jun, 2020 1 commit
-
-
Alex Ott authored
### What is this PR for? Fix a flaky test in SparQL interpreter ### What type of PR is it? Bug Fix ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4860 ### How should this be tested? * https://travis-ci.org/github/alexott/zeppelin/builds/695631580 Author: Alex Ott <alexott@gmail.com> Closes #3789 from alexott/ZEPPELIN-4860 and squashes the following commits: 19fb4ecb1 [Alex Ott] [ZEPPELIN-4860] Non-determenistic results in SparqlJenaEngineTest
-
- 05 Jun, 2020 1 commit
-
-
Jeff Zhang authored
### What is this PR for? Trivial PR which just make `zeppelin.spark.concurrentSQL` to be `true` by default, this is what user expect usually. ### What type of PR is it? [Improvement] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4840 ### How should this be tested? CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Jeff Zhang <zjffdu@apache.org> Closes #3783 from zjffdu/ZEPPELIN-4840 and squashes the following commits: 7ece99b83 [Jeff Zhang] [ZEPPELIN-4840]. Set zeppelin.spark.concurrentSQL to be true by default
-
- 01 Jun, 2020 3 commits
-
-
Alex Ott authored
### What is this PR for? When updating for the new DataStax Java driver, I forgot to update template for a `HELP` command. This PR fixes this problem, and simplifies maintenance by reading driver version from the resource file generated during compilation, so HELP command will provide information based on the actual driver version used for building interpreter. ### What type of PR is it? Bug Fix ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-4792 ### How should this be tested? * https://travis-ci.org/github/alexott/zeppelin/builds/693123862 () * Tested manually Author: Alex Ott <alexott@gmail.com> Closes #3780 from alexott/ZEPPELIN-4792 and squashes the following commits: f62bc5c51 [Alex Ott] [ZEPPELIN-4792] Help needs to be updated in Cassandra interpreter
-
Jeff Zhang authored
### What is this PR for? This PR will first set env `PYSPARK_PIN_THREAD` to be `true` and then suppress the warning message in `pyspark.util`. ### What type of PR is it? [Improvement ] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4802 ### How should this be tested? * CI pass * Manually tested ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? No * Does this needs documentation? Author: Jeff Zhang <zjffdu@apache.org> Closes #3778 from zjffdu/ZEPPELIN-4802 and squashes the following commits: d6a65cbde [Jeff Zhang] [ZEPPELIN-4802]. pyspark warnings with spark-3.0.0-preview - Sync of properties between JVM and PVM
-
Jeff Zhang authored
### What is this PR for? Trivial PR to set `zeppelin.spark.sql.stacktrace` to `true` as default value ### What type of PR is it? [Improvement] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-4836 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #3777 from zjffdu/ZEPPELIN-4836 and squashes the following commits: 547f1381f [Jeff Zhang] [ZEPPELIN-4836]. set zeppelin.spark.sql.stacktrace to true as default value
-
